U.S. DEPARTMENT OF COMMERCE 
National Technical Info-mation Service 


AD-A036 435 


AUTOMATIC TARGET HAND-OFF USING CORRELATION 
TECHNIQUES 


AuBurn UNIVERSITY 
ALABAMA 


31 JANUARY 1977 








7 : Pb By ASE 40S Kh AGE a ee ee ee ae ee - qe) AAs 





‘pate Bo ceoee 


“3 


ey wwe 


a OSS 


Ne, 
wooed 


tf 


Aviram Usiverety 
oe SOy R&* 


aghas 


B 


a \ Q 
oe aee bes 

ede Rae TE 

PP ASN 


et So 



















“a Pathe. 2 
as Saket si vf , 5 
shy BANG ae 2 
AN « : yee ' 
eT AR aE Pieters ‘ 
eo era 5 Fd, . ‘ 
ee SSS 2 SE 
z 
; 
. ’ 
; ‘ 
i : . os , 
} , 
: : i 
Sm ot Se ta “ty es \ 
neon ies SL a ts ato te fe ft Thad ot ISM age hale? c Be yo - 
Tye sper te ae OER ae [SBA ON cea. ‘ed * 


















mS yr e* regs TENE ey 
Pep g 


ate SR, 












OR 
a 
A 


we 
. 


“ 
3 


oe 
wy 


SAE, 
2) 
fo 
> 





















ee 
ne ate Anet 
SSS as FY ON TS 
Se eS ot 
wt wa Sy 















wee NS, 
Cee! 


a 









OT EE OE 















Rese ace ON 
The SL ERT ay, 
Re SINE, Hate eee ane Se 
IESG cee eke 

DN de NLS ha 8 
eho mete 










iS 












a4 
Me OT Re IN re NO, ce tre ae 





‘ 
ee 


rate iy THis ePonT ARE WOT TOES CONSTRUED AS AN 
SRSIAL DEPATTIANT. OF THE ARMY POSITION UNLESS 20 DESIG- 
SATES OV ONE AUTHORIZED OdeURtNTE. , 


atm, 





FRASS NAMES 


i. oe WAS CF WRASSE RANIS O23 MANUSASTONS (3 Ka VAS ROPONT BOIS 
: CAV GETTER OFF RAAL INGORA BENT OA ARPRGVAL GF 
a gs PA SES GF Nes WANA MARIIARS OR COSTWALS, 






AUTOMATIC TARGET HAND-OFF USING 
CORRELATION TECHNIQUES 


by 


J. S. Boland, III, L. J. Pinson, G. R. Kane 
M. A. Honnell, and E. G. Peters 
Electrical Engineering Department 


Auburn University . 
Auburn, Alabama 36830 


Final Technical Report 


For the Period 5 December 1975 - 31 January 1977 


This research work was supported by 
U.S. Army Missile Command 
Redstone Arsenal, Alabama 35809 
under Contract DAAHO1-76-C-0396 


ENGINEERING EXPERIMENT STATION 
Auburn University 
Auburn, Alabama 36830 


31 January 1977 


Cleared for Public Release; Distribution Unlimited 


/i 





DO DC 
Pee In te 
MAR 8 19ST? 


A mG 








eA NCR Ta Seen Ant DT RA lA LCE CRC CRORE: 


2 AR 


























SECURITY CLASSIFICATION OF THIS PAGE (When Dete Entered) 


REPORT DOCUMENTATION PAGE ation Coser etn poeil 


+ REPORT nUMOES GOVT ACCESSION NO] 2. RECIPIENT'S CATALOG NUMBER 


4&. TITLE (end Subtitie) 5. TYPE OF REPORT & PENMOCO CovEReO. 
Final Technical Report, 
5 Dec. 1975 - 3) Jan. 1977 


S. PERFORMING ORG. REPORT NUMBER 








AUTOMATIC TARGET HAND-OFF USING 
CORRELATION TECHNIQUES 






7. AUTHOR) T- CONTRACT OR GRANT NUMBER(S) 
J. S. Boland, III, L. J. Pinson, G. R. Kane, DAAHO1-76-C-0396 


M. A. Honnell, and £. G. Peters 


9. PERFORMING ORGANIZATION NAME AND ADDRESS 
Engineering Experiment Station 
Auburn University 
Auburr, AL 36830 

{t. CONTROLLING OFFICE NAME AND ADDRESS 12. REPORT DATE 
Commander, U.S. 4rmy Missiie Command 31 January 1977 
ATTN: DRSMI-RGC 19. NUMBER OF rary 


Redstone Arsenal, A 1809 
. MONITORING AGENCY NAME & ADORESS(:‘ different from Controiting Office) 1S. SECURITY CLASS. (of thie repert) 





UNCLASSIFIED 


Se. OECL ASSIFICATION DOWNGRADING 
SCHEDULE 












Cleared for Public Release; Distribution Unlimited 


17. OLSTRIGUTION STATEMENT (of the abstract entered in Block 20, {f different from Report) 


18. SUPPLEMENTARY NOTES 


19. KRY WORDS (Continue on reveree side If neceesary and identify by block number) 


Correlation, Target Hand-Off, Image Correlation 


20. ABSTRACT (Continue an reverse side if neceseary and identif, by block number) 


The problem of automatic haid-off of a target from a precision pointing 
and tracking system (PTS) to an imaging missile seeker is considered in this 
report. The approach taken is to search for the target in the seeke’ field 
of view (FOV) using the PTS video as a reference. When the target * located, 
the seeker line of sight is adjusted automatically such that the target is 
at the center of its FOV at which point the seeker tracker can lock on to the 
target. 





ND Ont, 1473 cornion oF t Nov 68 ts OBSOLETE 


SECURITY CLASSIFICATION OF THIS PAGE (When Date Entered) 


¢ 



















tem rt 

~. Plocation of the target in the seeker FOV can best be accomplished using 
correlation techniques. The approach taken is to consider the most accurate 
but yet most costly in computation time and hardware requirements. Trade- 
offs are then considered in order to obtain a real-time correlator (j.e., 
one which can compute the correlation surface at the rate of the incoming 
live video from the seeker). The effect of these trade-offs on correlation 
accuracy and other system performance criteria is given. A correlation 
algorithm is chosen and an implementation of this algorithm is given. An 
alternate implementation using an analog adder rather than a digital adder 
tree is recommended. 





TABLE OF CONTENTS 


Page 
LES TOF FAGURES coe ma 'y..0 a alone eee? ee seek er et Se ee 3 
LUST OF TABLES 3.5: i: si Se: oe ecient cae ach. a ta ee OR eg, Secs 7 
Te SENTRODUCTION: 255 2oscees ek i AE aw a SRE RUNS ee B 9 

As. “Scope Of Works.e 6.6 se aoe ae wee ee ee ad if 

B. Organization of Report .........0650 50205 ee 13 


2. FORMULATION AND COMPARISON OF CORRELATION ALGORITHMS... . 15 


A. Geometric Layout of PTS and Missile 
Seeker TV Images... 1. 2 ee ee we eee ee ewe 15 
B. Image Comparison Methods... 2... 2 eee eee eae 18 


Direct Method 

Fast Fourier Transform (FFT) Method 

Phase Correlation Method 

Sequential Similarity Detection Algorithm (SSDA) 


Oo 
> 
33 


alysis OF Iinage Comparison Methods... ...:2.2- 23 


Image Normalization 

Problems Associated with FFT and Phase : 
Correlation Methods 

Quantization of the Input Video Signals 

Image Normalization Using the One-Bit Quantizer 

Quantization Effects on S/N Patio for the 
Direct and SSDA Correlation Methods 

Implementation of the SSDA an‘ Direct Methods 
Using One-Bit Quantization 


3. CORRELATION SYSTEM CONSIDERATIONS. . 2... 6. 2 2 ee eee 53 
A. Determination of Sampling Frequency. .....-+- +s 53 
B.. Area Correlator: .. 6 6 a we Re we ee 55 
C. Preprocessing of Input Video. . 1... 1. ee eee eee 57 


High Resolution Image Preprocessinc 

Low Resolution Image Preprocessing 

Effect of Quantization about Incoriect Mean 
Preprocessing for siffering Sensor Characteristics 


] 





A pr ne EIEN, 


TABLE OF CONTENTS (Con't) 


Page 
D. Vibration Analysis. ... 1... ee we ee we we wens 71 
E. Derivation of aX and AY Misalignment in LR Image. .... 76 
F. Threshold Effects on Probability of False 
Registration anc Detection Probability. ......... 79 
False Alarm 
Detection 
G. Target Not in Seeker FOV. . 2... 1... 2. 2 ee wwe ne 97 
4c. IMPLEMENTATION io. 0:-& cc ae: eine See we we 101 
A. Implementation of One-Bit by One-Bit Real-Time 
Correlator........4..-. ix ei tote a! Gh ae wae Oe Cee a 103 
B. Block Diagram of 32 x 64 Array Correlator System. .... 121 
5. CONCLUSIONS 4° RECOMMENDATIONS . 2... 1 ew ee ee ee 131 
Rez. GONG LUST ONS: 6.4 8h at 6 seen ta Se ee Se SOL ee 131 
B. Objectives of a Technology Program. ........... 133 
REFERENCES ;. ‘c: ¢.-6: aS Yeas ecw: Boek. ew, OSS wel Pw 135 




















ont 
e 


wo © ~ n an > w ied 
. ° . . e e ° ° 


—_ ed 
ed OQ 
° . 


> 


—_ 
Nm 


13. 
14. 


16. 
Vs 
18. 
19. 


LIST OF FIGURES 


Page 


PTS equipment location on AH-1 helicopter. ........-> 10 
Target space projection of narrow and wide FOV images. . . . 16 
HR image projected onto LR image... ... 5. 2-2 es eevee 16 
K x LHR image located at position (p,q) of N x MLR image. 17 


Two functions to be correlated... .....2 2 ee eevee 19 
(MANET ZORS 6S ie S48 SE OR GS wR Se ee LS 29 
Six: Tevel ‘quantizers. c0:c: eee At AS ee we 3] 


Layout for correlation of first line of two images... .. 31 


Alternate one-bit quantizer... .......+6.602-+.2.2. 37 


One-bit correlator input/output tables. .......... 38 
Outcomes of |g)-I|,  - ee ee ee ee 40 
Comparison of the SSDA and Direct Double Summation 

MECNOGS.. <g: coo: oe ee Se ea a Be oe Be ke See 47 
Reversed polarity one-bit quantizer. ....... ee ee 50 
Input/output table for one-bit correlation. ........ 5] 
Exclusive-OR implementation... 1... 2.6 ee ee ewe 5] 
Preprocessing of HR image where W=3 .... 1.2.5. + ee ee 58 
Block diagram of preprocessor. ... 1... 2.2 ee eee e 58 
Preprocessing where W= 3.25. ........50+0 5040008 69 


Method for pixel quantization 


Quantizers for the HR and LR images 


eo © © © © © 2# © © © © 8 © 


j 
Be 
| 
4 
’ 





fate ae 


FRE ORES URE ERT 





LIST OF FIGURES (Con't) 


| 
| 
| 
| 
| 


Page 

21. Effect of dc quantization level on the variance 

in the estimate of p. 2... 1 wwe wee et tt ew 69 
22. Image layout identifying p* and q*. ..- 1... ++ 2. ee eee 77 i 
23. Image layout showing AX end AY... 2 2 2 2 eee ee eee 77 
24, Quantized signal plus noise. .........2-220-++08- 81 
25. Quantizer characteristics... 1... 0.2 eee ewe rene 81 
26. Correlation technique. ........0006 Be eid a: Se ete “Be 
27. Detection and false alarm probabilities versus 

decision threshold (KL=2048). ... 2.0.2... 4020+ eee 88 
28. Detection and false alarm probabilities versus 

decision threshold (KL=1024). .......2.26 cee eee 89 


29. Probability of detection versus probability of 
false alarm; parameter dependence. ..........+6-. 93 


30. Effect of SNR and pixel miswatch probability, ec, 
on probability of a correct decision for each 


DING] 6 eow. hs OE OO IE ROD BR Re ER Se 95 
31. Effect of different video SNR on probability of 

making correct decision for each pixel. .........-.-. 96 
32. Search patterns for missile seeker. ..........20240-6 98 
33. Block diagram of correlator hand-off system ......... 102 


34. Logic for 12 x 1 correlator... ......2.0 2 ew eee 104 
35. Extension of Figure 34 to produce a 12 x 16 correlator. . . . 105 
36. Logic diagram for a 15 x 1 one-bit correlator. ....... 107 


37. Extension of Figure 36 to produce a 15 x 16 correlator, . . . 108 


38;./ Items 1 Pipe «6 ow ak se a Se RS eB ee ES 110 
39. Correlator and adder tree... .. 2... 1 ee ee ee ene mW 
40. Memory requirement for correlation computation. ....... 115 


4 

















4}. 
42. 
43. 


44, 


45. 
46. 
47. 
48. 


49, 


LIST OF FIGURES (Con't) 


Method of computing mean video level. ........-4- 
Mean video computation and quantizer. ........... 


Sas for computing the maximum (i.e., the registration 
DOUG sheet cot GO Ee A Rn 0 DDR ee eee OP oh 


Picture size (P x Q) with horizontal and vertical retvace 
RANG: So se ap aoe Se a RE SN ca ee oh ed wie see 


Retrace (Edge) effects on quantization. .......... 
Binary noise source generator .........2-0e006-8 
Averaging circuit for resolution reduction of HR video. . . 


Blanking circuits for correlator during horizontal and 
vertical retrace. . 2... 6 1 wwe we ee ee ee ee 


Complete digital correlator circuit ...... oS Vibes tah 


Page 
115 


116 
118 


119 
120 
120 
122 


123 
125 


LIST OF TABLES 


1. Results of S/N ratio analysis 


2. Horizontal and vertical resolution ef 2.3 x 2.3 
mater target 


. *. y. - 
Fe ulate wepinenend set Aetpen 10 Asia ila EO 1128 10 HME ME ESS ad secieecdinetaeniiaaalal 


ee a ee eee a ee ee ee 56 
3. Angular disturbances in seeker LOS due to jitter ..... 75 
4. Comparison of ECL vs. TTL logic correlators ........ 106 2 
5.  One-bit by one-bit correlator using ECL logic. ...... 109 


. 
. 
. 
2 
. 
° 
. 
. 
° 
. 
. 
= 
a 
pte Sale Ud 


Preceding page blank 


GL 


x TEER ESTA ES FS SA A Ae Te 
LGR ERS OOF, AES F = 
Ca = meneame tee ania = 


ere remo 





1, INTRODUCTION 


Currently the U. S. Army is developing a system for the acquisition, 
tracking, and laser designation of military targets fruin heliccpters. 
The Army is also developing terminal homing guided weapons capable of 
destroying designated military targets. The Army is also beginning to 
investigate misciies which have imaging seekers rather than laser seekers 
to provide guidance. This provides a true fire and forget weapor :ystem. 
Very little attention has been given to-date to the problem of hand-off 
of a target from the acquisition system to the imaging missile seeker. 
This task can be accomplished by the gunner using the video displey from 
both systems on his monitor. However, this technique is time consuming 
resulting in increased helicopter exposure time. As a result the U. S. 


Army Missile Command, Huntsville, Alabama let a contract with the 


etsy 
ee ne te a I AR AT ROA AAAS AE REE CAEN AATEC ? 


Engineering Experiment Station, Auburn University to study and make a 
recommendation for a system to accomplish automatic hand-off of targets 
from designators to imaging missile seekers using correlation techniques. | 
This report presents the result of that effort. 

The overall system consists of the helicopter with pilot and 
gunner displays and controis, the pointina and tracking system (PTS), 
either integrally mounted or on the stores wing of the helicopter as 
shown in Figure 1, and the missiles located in a stores rack also 
mounted on the wing of the helicopter. The helicopter supplies power 
and other electronic support hardware for both systems. The pointing 
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and tracking system typically consists of an optics train, line-of-sight 
(LOS) stabilization system, day TV and forward tonking infra-red (FLIR) 
imaging systems, manual and autotrack system, laser range finder, and 
associated electrcnics section. The imaging missile seeker could be 

a day TV system or a FLIR. Tne current advanced television seeker 
(ATVS) being tested by the Army [1] is composed of optical, day tele- 
vision camera, TY tracker, mode control, stabilization and power supply 


subsystems. 


GUNNER'S CONTROL 
CONSOLE; VIDEO 
MONITOR, HAND 
CONTROL, CONTROLS/ 7. 


INDICATORS / 


PILOTS INDICATOR 
RANGE AND GIMBAL 
ANGLE DISPLAYS 










PTS POD 
ELECTROMICS AND ies, 


TURRET ASSEMBLY 


AMMUNITION BAY 
MOUNTED EQUIPMENT 


Figure 1. PTS equipment location on AH-1 helicopter. 
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In order to accomplish hand-off using correlation techniques, 
the two signals being correlated must be similar. If both the PTS 
and missile seeker are day TV systems sensitive in the same rela- 
tive optical spectrum, correlation can be accomplished with very 
little signal preprocessing. However, if the two signals are 
dissimiliar such as from a day TV and FLIR, then correlation possibly 
can be accomplisned but only after considerable preprocessing of the 
signals. For example, correlation on edges alone might work in this 
case. The correlator developed in this report is for two day - 
TY sensors. The problem of dissimilar sensors is discussed, however, 


in Chapter III. 


A. Scope of Work 
The scope of work performed under this contract was tc analyze and 
make recommendations on the hardware and software required to be used 
in "handing-off" a target from a target acquisition device to: an imaging 
missile seeker. The function of an automatic correlator is to compare 
the video signals from the high resolution pointing and tracking system 
(PTS) and from the missile seeker and to generate error signals which 
can be used to drive the missive seeker such that its aimpoint coincides | 
with that of the PTS. The scope of work as written in the contract | 
is given in the following six paragraphs for completeness. 
The first phase will be to choose one or more candidate correlation 
i 
i 


approaches. This will be done by surveying the literature for information 


on existing correlation techniques. It is realized that some difficulty 
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12 
may arise since most companies have developed correlation techniques 


using their own funds and therefore consider them proprietary. It 

is likely, however, that some information on general correlation 

techniques is available in the open literature. 

: Along with such information as is available on existing techniques, 

: an independent solution to the “hand-off" problem will be proposed 

considering the variables involved (i.e., fields-of view, system 

resolutions, stabilization, and others). The approach wil! be 

to consider first the most accurate and complex correlation technique 

and propose trade-offs in Phase Three. 

j Since the target may not necessarily be in the seeker field of 

7 view (FOV), an algorithm will be proposed to slew the seeker TV until 
the target is in its FOV. Of course, 1t is expected that this operation 
will be very time consuming and should be avoided if possible. One 
possible alternative is to determine and store in the fire control 
computer boresight errors while in flight before a target is engaged. 

4 The trade-off of additional software and computer memory required by 

, these two methods will be considered. 

Phase Two, which will actually be conducted concurrently with 

Phase One, will consist of on investigation of the hardware and software 

; needed for various combinations of PTS and missite seeker configurations. 
Consideration will be given to differing fields-of-view, scan rates, 
stabilization of TV cameras, boresight, and TV format (such as aspect 
ratio, frames per second, etc.). The contractor will consider various 


: combinations and recommend the hardware and software necessary to 


splement each. 











13 
Phase Three will consist of the actual analysis of complexity 


of the hardware/software necessary to implement the various combinations 

of Phase Two. This analysis will be presented in such a way that 

MICOM will be able to use it to maxe a trade-off of the various 

options available (e.g., complex/versatile vs. simple/closely bounded). 
In Phase Four a correlation technique, along with the system 

constraints, wili be chosen in concert with the MICOM technical 

director. This cecision will be based on the results of the first 

three phases. The hardware and software to implement this approach 

will be completely specified, and the correlation algorithm wil] 

be developed. 


B. Organization of Report 


All of the work outlined in the Scope of Work has been completed 
and is documented in this final report. Chapter two formulates the 
hand-off problem in greater detail and gives the various correlation 
techniques considered. As expected, all companies contacted either 
did not respond or gave very little detaii ‘ints their correlation 
algorithms. As a result, the effort on this contract started with 
a literature search of correlation techniques, defining the most 
accurate but most time consuming methods, and making trade-offs in 
order to arrive at an implementable solution. 

Chapter three goes into detail on the many correlation system 
considerations which had to be resolved before an implementabie solution 


could be attained. These considerations range from the system optics, 


s 
af 
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TV sensor and TV format, electronic preprocessing, sampling rates, 


two system resolutions, vibration analysis, slaving of two systems 
before correlation, derivation of error signals to drive the seeker 
gimbals, and others. 

Two possible implementations of the correlation algorithm are 
given in Chapter four. The first implementation was considered to be 
the state-of-the-art up until October when TRW publicly announed their 
64 bit, 20 MHz bipolar correlator LSI circuit. The second implementation 
incorporates this TRW chip and significantly simplifies the 
hardware. The reason for giving both implementations is to emphasize 
how LSI state-of-the-art can effect the real-time correlation problem. 
It is known that TRW, and possibly other companies, is working on 
another LSI correlator chip which will simplify the implementation 
further or make possible an implementation which now is not feasible 
for real-time correlation. 

Chapter five gives the conclusions and recommendations resulting 
from the above work. Questions which arose during the course of this 
work but were not answered due to lack of time or lack of simulation 


facilities are also given. 





2. FORMULATION AND COMPARISON OF CORRELATION ALGORITHMS 


In this chapter the basic correlation problem is formulated, four 
methods are presented to solve the problem, and finally a comparison 
of the four methods is given. Two of the methods are shown to be 
impractical at the present state-of-the-art because of computational 


burden. 


A. Geometric Layout of PTS and Missile Seeker TV images 
For the purpose of designing a correlator, this report will 

consider hypothetical PTS and missile seeker day TV systems. To 

design a correlator for a particular PTS and missile seeker, some of 

the design parameters will have to be changed but the basic procedure 

will be identical. Consider a PTS system with a vertical narrow field 

of view (NFOV) of .5°. a 525 line standard TV system with 2:1 interlace, 

60 Hz field rate, or 30 Hz frame rate, and a 1:1] aspect ratio. The 

sensor TV will be assumed to have a 2° vertical FOV with 4:3 aspect ratic, 

a 525 line standard TV system with 2:1 interlace 60 Hz field rate, 

or 30 Hz frame rate. Both systems because of vertical retrace will | 

be considered to have 480 active TV lines. 
One of the fundamental requirements for correlation using any of 

the methods of the next section is that the two TV images be preprocessed 

such that the scenes have the same spatial resolution. Since both TV | 

systems have 480 active TV lines, the PTS will have 4 times as many 


lines on a given target as the missile seeker TV because of the 1:4 FOV 


ratio as shown in Figures 2 and 3. Since the resolution of the seeker 
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16 
TV cannot be increased, the obvious solution is to decrease the PTS 


resolution ty a factor of four. This can be accomplished by averaging 
the PTS video as shown in Chapter III. For the remainder of this chapter 
it will be assumed that the PTS image, henceforth referred to as the 

high resolution (HR) image, and the seeker image, henceforth referred 

to as the tow resolution (LR) image have been preprocessed such that 

they have the same spatial resolution prior to application of any 


correlation algorithm. 


| Th 480 Active 


Lines 480 Active Lines 4a 


High Low 
peeclutiah Resolution 
Image Image 

(WR) (LR} 





Figure 2. Target space projection of narrow & wide FOV images. 





Figure 3. HR image projected onto LR image. 











V7 
After the resolutions of the two images are equalized a number of 


correlation or matching methods can be investigated. For the remainder 
of this report the dimensional relationships between the two images 


will be as shown in Ficure 4. 


$ (9.4) = (04+) 
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Figure 4. Kx LHR image located at position (p,q) of Nx MLR image. 


The missile seeker image is represented by a N X M array of points. 
The values of N and M would be determined from a choice of sampling 
rate and number of TV lines of the missile seeker system. The PTS image 
or HR image is represented by a K X L array of points in Figure 4. This 
K X L array might be all or only a portion, containing the target, of 
the PTS image after its resolution has been changed to that of the missile 
seeker image. The p and q dimensions in Figure 4 give the vertical and 


horizontal position of the HR image in the LR image. These indices 


Start in the upper left corner of the LR image, where p = q = 0. 
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B. Image Comparison Methods 


For digital matching of two images a number of alternatives 
exist. In this section four candidate algorithms are presented and 
explained. A detailed comparison of these methods is given in 
Section C of this chapter. 


1. Direct Method 





The classical approach to the problem of determining where twc 
signals match is correlation. Consider two functions f, (t) and 
fo (t) as shown in Fiqure 5. The correlation integral is defined to 
be : 
C(t) “J Fi(t) fo(t + T)dt (1) 


where T is allowed to take on values between -o and +o, “he value of 
tT which maximizes C(t) in Equation 1] is the correlation peak and is 


defined to be the match point between the two signals. In Figure 5 





this will occur when T = ty - ty; It is obvious that determining the 
correlation peak consists of multiplying one signal by the other signal 


shifted by T and then evaluating the area under the resulting curve. 
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Figure 5. Two functions to be correlated. 


If both signals are sampied and held, then Equation 1 can he 


approximated by the following equation. 


o 


C(p)=T J Fy(n)fy(n + p} (2) 


Q=~e 


where f(n) = f, (nT) and fo(n +p) = f(nT + pT) and T is the sampling 


interval. As T becomes very small, the result in Equation 2 approaches 
that of Equation 1 where the p which maximizes C(p) is defined as the 


co.relation point. 
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The two TV images are first sampled and preprocessed to match 


spatial resolution and then stored in arrays. The HR image is aK KL 
array and the LR image ‘s a N X M array as shown in Figure 4. Therefore 


a two-dimensional discrete correlation algorithm is given by 


tb 3 
R(p.q) = fy EE (nam) Lh(ntpsat) 


for O<p<N-X« 
Q<q<M-L (3) 


where R(p,q) is the correlation function, and the division by KL 
is a scaling factor. Equation 3 is referred to as the Direct Method in 
this report. 

Using the algorithm of Equation 3, the selected K X L array of 
HR points is compared to each array of LR points of dimension K XL 
in the totai N X MLR array. The algorithm produces the correlation 
array R(p.q). In most situations the maximum value of the correlation 
function indicates image registration or match. However, in the 
present case since the 12 image spans a wider field-of-view, R(p.q) 
is actually a cross-correlation, and therefore it is possible that the 
maximum value of the correlation functior does not indicate a target 
match between the HR and LR image. In order that the maximum value 
of the correla:ion function indicate target location in the LR image, 
both image array values need to be normalized. Once normalization is 
accomplished, the calculation of correlation points is insensitive to 
magnitude values and depends on the pattern of data points found in 


the K X L arrays of the LR image. Therefore, in order to use the Direct 


Method, image normalization must be implemented. Image normalization is 
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discussed in further detail in Section C of this chapter. 


2. Fast Fourier Transform (FFT) Method 


Because of the transform relationship hatween correlation and 
power spectral density, the cross cerrelation between the HR and LR 


images is given by 


R(p.q) = IFFT {|FFY SHR(n,m)] FFT [LR(n,m)}]3 


IFFT (G(k,1)} (4) 


where FFT [- ] is the fast fourier transform, IFFT is the inverse fast 
fourier transform, G(k,1) is the cross power spectral density of HR 

and LR and R(p,q) is the cross correlation function. In Equation 4 
both the HR and LR arrays have to be the same size and preferably 

be a power of two. To accomplish this both arrays are filled out with 
zeroes to form 512 x 512 arrays. The result yields 512 x 512 values of 


(p,q). 
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4. Phase Correlation Method 


A major problem in correiating two different size images is that the 
resultant correlation function is in fact a cross-correlation instead of 
the desired auto-correlation. The auto-correlation function has its 
maximum value for zero relative shift; this is not necessarily true for 


the cross-correlation function. Thus it is quite possible that the 
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cross-correlation function will have false peaks which are larger than 


the peak for zero shift. The false peak condition can occur quite easily 
for portions of the LR image of high brightness, especially if the 
smaller HR image is of lesser brightness. Therefore, the problem is 
one which depends on relative magnitudes of the various sub-elements of 
the larger image. 

One technique that reduces magnitude effeczs in *** zross-correlatien 
function is phase correlation. The phase correlation f.action between the 


HR and LR images is given ty 


a(p.q) = rrr | SEFTLWR(nam)) TFFTELR(nan s 
{ [{FFTCHR(n.m)]} {FFT[LLR(n,m)}} | 


where * denotes the conjugate. 

The phase correlation method is essentially a point-by-point 
normalization in the frequency domain of the cross-power spectrum. 
The same requirements on the size of the HR and LR arrays exists as 


for the FFT method. 


4. Sequential Similarity Detectian Algorithm (SSDA 


Another algorithm for determining the similarity and amount of 
registration between two images is known as a Sequential Similarity 
Detection Algorithm (SSDA). Consider a K X L HR image (reference 
image) and a N X MLR image. The SSDA registration surface is 


comutea by adding the absolute value of the difference between the 


HR and Lk pixels for each value of p and q as given by Equation 6. 
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E{p.q) = 2, Ey [HR(n.m) - LR (n+p, meq)| (6) 


for O0< p< N-K 
O< q< M-L 


The registration point (denoted by p*, q*) is the value of p and q for 
which E is a minimum. The advantage of this method over the Direct 

Method is that subtraction circuits rather than multiplication circuits 
are needed for implementation. It should be pointed out, however, that 
the SSDA method is not a correlation technique since Equation 6 is 

not related to Equation 1, the correlation integral. A further comparison 


of these two methods is given in later sections of this report. 


C. Analysis of Image Comparison Methods 


1. Image Normalization 


As stated in Section B of this chapter it is necessary to normalize 
the correlation algorithms so that false peaks are less likely to occur. 
False peaks can occur for the following reasons: 

a. For correlation computations using the Fast Fourier Transform 
(FFT), the two arrays must be the same size. To accomplish this the 
HR array is filled in with zeroes to make it the same size as the LR 
array (array sizes must be a power of two). Therefore the two images 
being correlated are not identical which means the computation is a 
cross-correlation rather than an auto-correlation. There is more of a 


possibility for false peaks (any peak which does not correspond to 


correct registration) in cress-correlation than in auto-correlation. 
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b. The size of the target in the HR array may be slightly 


different from its size in the LR array. This could be caused 
by the ratio cf the field of views (FOV) being slightly different 
from the assumed ratio or by the FOV ratio not being an integer 
number. This reduces the magnitude of the peak at the actual 
correlation point which could result in an erroneous decision. 

c. Contrast differences in the two TV signals could cause a 


false peak. 


d. Differences in illumination levels can cause false peaks. 


To illustrate this consider the Direct Method given in Equation 3. 


In the ideal case of exact correlation each LR pixel is exactly 


equal to the corresponding HR pixel 
1 & £2 
R(p*, q*) =— fF HRS\ i,m) 
KL n=] m=? 


where p* and q* denote the coordinates for perfect registration. 
For this ideal case consider a non-matching coordinate point Dp, q 
where 


LR(n+p,mtq) = max HR(n,m) = HR ax 


Then 


aA 1 =&k L 
R(P.q4) = neyo HR(n,m)HR_.. > R (p*.q*) 


It is easily seen that even in the perfect registration case, a search 


for the peak value of R(p,q) does not necessarily yield the correct 


registration point. For the Direct Method, normalization can be 


accomplished as follows: 


(7) 


(8) 
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Es E__ HIR(n,m)LR(n+p meq) 
R(p,q) = Se=L wl (10) 
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E r HR ina] | r : Uo sta] 
n 


=] m=] n=] m= 


This obviously involves considerable more computation time than the 
unnormalized Direct Method cf Equation 3. 
For the FFT Method, normalization can be accomplished by evaluating 


a enaammennanmennenenenemnnanatl 
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the IFFT of the phase correlation function as outlined in Section B-3 of 
this chapter. 
The Sequential Similarity Detection Algorithm (SSDA) can be normalized 


by changing Equation 6 in Section B-4 of this chapter to 
k oL as, 
E(p.q) = : fF [HR(nym) - AR - LR(ntp, meq) + LR(p.q)| = (11) 
n=] m=} 
for O< p< N-K 
O< as ML 
where LR(p,q) and HR are average shades of grey and are given by 
os » kt 
LR(p.q) =~ «FE LR(ntp, meq) 
KL on=1 mel 
(12) | 
a= gf HRInm) | 
HR=—— £ 5 HR(n,m i 
KL n=) m=} ; 
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2. Problems Associated with FFT and Phase Correlation Methods 








Of the four methods presented in Section B, the FFT and Phase 
Correlation methods are the most difficult to implement. A number 
of factors involved with these two sethods make them somewhat less 
than prime candidates for the hand-off problem. Both methods require 


eo ere nerennnrnneterrE mer ae 


that the input arrays be square and that the dimension should be a 
power of two. As shown in Section A, the HR image is preprocessed 
because of its smaller FOV such that it is no longer the same 
dimension as the LR array. Therefore it would be necessary to fill the 
HR array with zeroes in order to make it the same size as the LR 
array. 

Since it is not known where the target may lie in the LR image, 
it is required that the entire LR array be stored when using the 
FFT or Phase Correlation methods. The HR image a?so has to be stored 
in memory. Additional memory is also needed for the intermediate | 
steps of these methods. The Direct Method and SSDA rethods can be 
used with only a small percentage of the memory needed for the FFT 
and Phase Correlation methods. Also, both the FFT and Phase Correlation 
algorithms involve, at one step, complex multiplications of two- 
dimensional arrays. ‘his again would add greatly to the hardware needed 


for their implementation that is not needed for the direct or SSDA methods. 





Because of the increased amount of hardware required and the fact that 
the FFT and Phase Correla 7on methods cannot be implemented in real time, 
the FFT and Phase Correlation methods were eliminated as candidate algorithms. 
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3. Quantization of the Input Video Signals 


All the algorithms discussed in Section & of this chapter require 
that both the HR and LR video signals be digitized. The process of 
digitizing continuous signals can be thought of as two separate steps. 
The first is sampling at discrete instants of time and the second is 
quantization. A discussion of the selection of an appropriate sampling 
rate will be given in Chapters 3 and 4 of this‘report. In this section 
the different quantizers investigated are discussed and the trade-offs 
used in choosing a particular quantizer are given. 

The input-output relationship for the four basic quantizers con- 
sidered in this section are shown in Figure 6. Figure 6 (a) is the 
multibit quantizer or one that has a very large number of quantization 
levels. Since there are many fairly inexpensive commercially available 
8 bit A/D converters capable of operating at. 10 MHz, they will? be con- 
sidered as multibit quantizers. 

Figure 6 (b) gives the input-output relationship of a tro level 
or one-bit quantizer. The weight of the output is +] if the input is 
above the mean voltage level and is -] if the input is below the mean 
voltage level. With this quantizer a means must be provided for 
finding the average shade of grey of the K X L high resolution reference 


image and of each of the K X L sub-arrays of the low resolution image 
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corresponding to each set of p and q values in Equation 3. This 


could be dene by quantizing the images with an 8 bit A/D ccnverter, 
computing the average values in the host computer, during one frame, 
and using these computed average values for one-bit quantization 
during the following frame. The average signal level is not expected 
to change significantly from frame-to-frame. If the average signal 
level does change significantly from frame-to-frame, some form of 
adaptive average computation should be employed. A sub-optimal but 
real-time implementable solution to this problem is given in Chapter 
2 

Figure 6 (c) is the input-output relationship for a four level or 
two-bit quantizer. Assuming the mean and standard deviation of the 
input signal are known, there are two parameters, "a" and "Vo/o"s which 
can be chosen so as to minimize the variance of the estimate of 
R (i.4., minimize the loss in S/N ratio due to quantization). 

Figure 6 (d) is the input-output relationship for a three level 
quantizer. The advantage of this quantizer over the two-bit quantizer 
is that the weighting of the products in Equation 3 are either + 1 or 
0 which leads to a simpler circuit realization. The disadvantage is that 
the luss in S/N ratio is greater than for the two-bit quantizer. The 
mean and standard deviation (rms value) of the input video must be 
calculated for this method. 

The selection of the optimum parameters, a and Vo/o is made from 
an analysis of the degradation, in the mean square S/N ratio. This 


analysis is presented in Section C-5 of this chapter. 
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c) Two-bit quantizer d) Three level quantizer 


Figure 6. Quantizers 


Higher order quantizers are not considered since they would require 
larger adders with larger computation times thereby leading to smaller 
arrays for real-time correlation. It will be shown in a later section 
that the S/N ratio gain from using a larger reference array is greater 
than the gain using higher-order logic in the quantizers in a smaller 


reference array. 
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4. Image Normalization Using the One-Bit Quantizer 


| 
it 1s shown in Section C-1 that the Direct Method requires 
normalization to prevent false peaks. A method of normalization 

is given in Equation 10. When using the transfer function shown in 

Figure 6 (b) to quantize both the HR and LR signals, the outputs | 
are always + 1. Under this situation the denominator term of Equation 


10 reduces to (x.)?. Then Equation 10 becomes 


R(p.q) = —L, : 5 ran. e(nrma)| (13) 
(KL)? [net met 

Equation 13 is the normalized direct cross-correlation expression for 

one-bit by one-bit quantization. The square root of Equation 13 would 

also be a normalized estimate of the cross correlation and is seen to be 

equivalent to Equation 3. Therefore the one-bit by one-bit two- 

dimensional correlation algorithm given by Equation 3 is normalized. 

Its peak value, which occurs at complete registration, is one. No 


additional hardware dedicated to the normalization process is required. 


5. quant zation Effects on S/N Ratio for the Direct and SSDA 
orrelation Methods 


A detailed analytical study was performed so that the effect on the 
mean square S/N ratio due to different quantizers being used on the HR 


and LR signals could be determined. The analysis was done for the 
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quantizers shown in Figure 6 and also for the six level quantizer shown 
in Figure 7. 


Input 





tigure 7. Six level quantizer. 


In this section the variance derivations for the Direct and SSDA methods 
using one-bit quantization are presented. The results for the other quanti- 


zers are summarized and a comparison of the two methods is presented. 


a. Direct Methed 


Consider the estimation of the Direct Cross-Correlation function 
for the first line of the high resolution image with the first line of 


the low resolution image as shown in Figure 8. The estimation of the 


: i : 
a: Bs ieee 


Figure 8. Layout for correlation of first line of two images. 
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Direct Cross-Correlation function for the first line is given by 


R(1,q) = r HR(1,m)LR(1, mq) (14) 


where HR and LR are the high resolution and low resolution images 
respectively. To simpiify the notation, the first row designation can 
be dropped yielding 


: L 
R(q) =} J, 2RCn)LR(mg) (15) 
m= 


If the sampling rate is less than or equal to the Nyquist rate, the 





except for the three or four values of q close to registration, the 


adjacent samples can be considered to be indenendent. Therefore, 
correlation coefficient between the two images is small (i.e., » << 1). 
! 
} 


If the two images are quantized with quantizers 9 and 9, respectively, 


then Equation 15 becomes | 


% L 
Ra) = { J, 9 LARC) gCtR(meg)] (16) 
m= 
The variance of this estimate is given by 


on -G2> - <a> (17) 


where <R> denotes the expected value of R given by 


P L 
<R> = E t } (CLR) (18) 
m=] 
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However, because of the assumed independence of the samples, Equation 


18 reduces to 


L 
Be | 
cs =} ECoiR(w)] —goftn(mea)] > (19) 


The first term in Equation 17 can be written as 


‘ LoL 
<R’> <b L L g, CHR(m) }go[LR(m+q) Jg, [HR(n) J 
n=} m= 


Consider the case where both quantizers 9) and oP) have an input/ 


L 
(20) 
sgp{tR(ntq)) > 
Separating the term for which n=m in Equation 20 yields | 
: L 
&>= } oy “1n) 9,27 RUmva)} > | 
lL” om=) | 
1 L L tf \ 
oe deal € 9,LHR(m) JER (ra) 1 LARCH) T9pULR(nea)] > 
mgn (21) 
| 


output relationship as shown in Figure 6(b), then Equation 19 becomes 


ia: sas 20 2S 
> = 1 . 
R= I t f f (1) (HR,LR)d(HR)d(LR) J / (1) 


0 oo 
f(HR,LR)d(HR)d(LR) + f f (-1)f(HR,LR)d(HR)d(LR) + 
0 


oo 0 
f f 1am Lada) (22) 
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where f(HR,LR) is the joint density function of the HR and LR images. 
If #(HR,LR) is symmetric with zero mean, then Equation 22 reduces to 


4s L 
<P> = i L C(1)(1/4) + (1}(1/4) # (-1)(174) + (-1)(1/4)) 


or <R> = 0 (23) 


Because of the assumed independence of the samples and the input/ 
output relationship of Figure 6(b), the first term in Equation 21 reduces 


to 
L imal (24) 
_ L™ sm] a 
and the second term in Equation 27 becomes 
] . 2 2 
(L)(L-1) GG,L¥R])>? GpELR}> ? = 0 (25) 
5 ifn] >? GptuR)> 
Substituting Equations 23, 24, and 25 into Equation 17 yields 
2,1 
on i (26) 


Equation 26 is an expression for the variance of the estimate 
of R when the input is quantized into two levels. To obtain the 
variance of the estimate of the true correlation of the two signals, 
as the relationship between R and p must be established. If the 
input/output relationship for the quantized high resolution and 
quantized low resolution video signals are given by g, (HR) and g.,(LR) 
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respectively, where 9, and g, are both as show: in Figure 6(b), then 


Ria) = Coy{Hlgpf.R¢4)] > (27) 


where the HR and ik(q) signals are jointly normal with covariance 1 


given by 


and correlation coefficient p given by 
2 —H_ (29) 
SHROLR 


If x and Y are jointly normal random variables with covariance » 
and the random variable Z is formed by Z = g(X, Y) where g(x,y) is an 


arbitrary function, then 





E(Z} = E(g(X,Y)} = f f g(xsy)f(x.y)dxdy (30) 


where f(x,y) is the bivariate distribution function for the random 
variables X and Y. Price's theorem [2], [3] states that 


aME(Q(KY)} « aah | 
au” ax''ay" 
(31) 


o oo non 
-[ a” a(x,y) iX2y f(x,y)dxdy 


ax ay 





os ir LR(q)> - <r CiR(q)> (28) 
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Applying Price's theorem with n=] to the expression for R{q) in Equation 
27 yields 


ety” 


Se a 


a |) | dHR te (32) 


and from Equation 29. 


aR(q) . aR(q) , an(q) . aR 
ania} ay autat a)” aug) CHR OLR (33) 


or 
ag, [HR] ago[LR(q)] 
ona} = On ODE one 2 (34) 
30(q HR “LR dHR aLR(q) 
However from Figure 6(b) 
29) 895 
sak = 26[HR] and aa = 26[LR] (35) 


and Equation 34 reduces to 


ak(a} “ ft f ; f ; 2s[HR]26(LR] 0 





2 2 
1 = HR 2 HLA + ER OY | acurya(Le) 
” 2(1-p°) HR LR op 


or aRiq) . 2 (36) 
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where it has been assumed that HR and LR are jointly gaussian. For the 


case being considered p << 1 which leads to 
Rad z= 2 (37) 
el " 


From Equation 37 


On Oa k (38) 
POne-Bit Rone-Bit 


2 

ss 3.2 e (7) (39) 

: 4\C 
One-Bit 


we 


using Equation 26. 
If one uses the two level or one-bit quantizer with input/output 


relationship as shown in Figure 9 and repeats the steps given above, one 


finds the variance of this correlator to be 


» 2 
2. 1 
= 3(%)(7) oe 


or a variance which is exactly three times that obtained using the one- 
bit quantizer described by Figure 6(b). If the quantizer in Figure 
6(b) quantizes to levels of +n, repeating the steps above reveals that 
the variance of the resulting correlator is again given by Equation 39. 


Output 


Input 


Figure 9. Alternate one-bit quantizer, 
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The products for the correlator described by Figure 6(b) are either 
-1 or +1 as shown in Figure 10(a). The correlator described by Figure 9 
with an input/output table given in Figure 10(b) yields an estimate of p 
with a variance three times as large as the correlator in Figure 10(a). 
If one biases the output products of the correlator described in Figure 
10(a) as Shown in Figure 10({c), the mean of the estimate changes out the 
variance does not. The correlator in Figure 10(c) is therefore equivalent 
to that of Figure 10(a). Now if one implements the correlator in Figure 
10(d), the resulting variance is the same as the correlator in Figure 
10(c) but the counter can be designed to count only half as high thereby 
simplifying the implementation. The correlator with inpuS/output rela- 
tionship described in Figure 10{d) is easier to implement than the one 


described in Figure 10(a) since a ripple counter rather than an up-down 








counter can be used. In addition the input/output relationship described 





in Figure 10(d) can be realized with exclusive OR gates. 








0 1 
-] 0 0 
+] 1 Po) 
a) One-bit correlator, b) One-bit correlator 


with biased input, 





c) One-bit correlator d) One-bit correlator 


: 


with biased output. equivalent to (a), 


Figure 10. One-bit correlator input/output tabies. 
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b. SSDA Method 


Consider the estimation of the “SDA function for the first line of 
the high resetution image with the first line of the low resolution 
image as shown in Figure 8. The estimation equation for the SSDA 
function is given by Equation 41 where HR and LR refer to high resolution 


and low resolution images respectively. 


r L 
R(1ea) = [IWRC Tom) — LRT, ma) (41) 
m= 
To simplify notation, the first row designation can be dropped, then 


Equation 41 becomes 


; L 
Ra) = TJ, IHR(m) ~ Lr(neg) (42) 


Assume that the sampling rate is less than or equal to the Nyquist 
rate. This means that adjacent samples are almost independent for all 
values of q except for values near the point of registration. Therefore 
for all cases, except the case near registration, the correlation 
coefficient, p, is smal] and assumed to be much less than one. 

If the two sampled lines of video are quantized with quantizers 9 


and Io respectively, then Equation 42 is written as 


“ L 
R(q) = ¢ }, I9yCHR(m)] - gpCLR(méq)) (43) 


m= 
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ee Te SITRRES OF AIVTeT ENE quantizetton Methods can be investigated by 
considering the cases of q for which the samples can be assumed inde- 
pendent. The variance of the estimate of R is given by 


Mt 





0.2 = <R2> - <>? (44) 
R 


Assume that the quantizing functions 9 and Gp are given as in Figure 9. 


Consider first <R>, given by Equation 45. 


x L 
cis =<f} Foy ind] - gpftrinsa)iit> (45) 


Since the samples are independent, Equation 45 becomes 


r= NI IN Ee SEL O PEER ETT AOR Te 


‘ L 
cf T 2, <toyCoe(n)) ~ Gof LR(m+q))|> (46) 


Figure 11 shows the possibile outcomes of the function |g, CHR(m)] - GoLLR(m+q)]| 


for all combinations that the values 9} and J can possess. 


a, LHR(m)] 
0 ] 


0 
Gof LR(m+q)] 
1 


Figure 11. Outcomes of |g, - gol, 


Using the results of Figure] and expanding the expected value, 


Equation 46 becomes 





4) 


0 > 

: L 

ct =} j | | I (1) € (HR, LA (HR)G(LR) 
mzl[/-0 J0 


- 70 
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Assuring HR and LR to be jointly gaussian and assuming independent 


(47) 


samples, Equation 47 yields, 


oy L 0 fe 
<R> = c a {2 f I rma) aint. 


eR> = } (L)(2) (7) - (48) 
Therefore, the second term in Equation 44 is 

cho? =} (49) 
Next consider <Ro> given by Equation 50 


i, ik 
2 1 
<R™> = HR - g.{LR . 
<ts sey pep LanEHROm)] ~ gpCLR mea) I] 


(50) 
|g, CHR(n)} - aust 


Separating terms for which m=n in Equation 50 yields 
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we] 


7 & & 
ve ey adh <{Is, [HR(m)] - go[LR(m+q)]|° (51) 
mn 


[9,LAR(n)]~ gpfLR(mq)I|}> 


Again using the results of Figure ll and the assumptions made previously, 
Equation 5] becomes 


0 © 
21 t 
oe 3 f f (1)#(HR,LR)d(HR)d(LR) + 
LoL 
f f (1) (HR, upacmen| 2 b ), 


wish . 


If f (1) F(HR,LR)d(HR)d(LR) + f f FUR, navman 
sf f o 0 

(1)F(HR,LR)d(HR)d(LR) + nrimunacnan| 
LS, Sf 


(52) 


Equation 52 reduces to, 


d= () we (1) + (Ix) wore (f) (2 (9) 


(53) 
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Substituting Equations 49 and 53 into 44 yields 


2.1 ,1_1,1 
% era 


(54) 


Of importance here is to find out how the mean square Signal-to-Noise 


{ms S/N) ratio is degraded by the use of different quantizers. The 


variance of the estimate of p, 0°, directly reflects the degradation. 


The variance of p is found as follows. 


Let 


r(q) = E{|g,fHR(m)] - goLLR(mq )1]) 


Assume that HR and LR have the joint gaussian density function given by 


Equation 56 


f(HR,LR) = a 


Then Equation 55 is written as 


(55) 


2 2 
uelccal HR" _ 2pHRLR , LR 
Fe (; 2” OHROLR sa) 


ra —t ff" I “|, CHR(m)] ~ g5[LR(m+q) | 


one 


oles] d(HR)d(LR) 


Using the results of Figure 1] and assuming ae 


(57) 


<< ], Equation 57 becomes 


mo lS 


aa 


we 


ae aaa 


ECs LR wr) | 
0 fe HR LR 
r(q) teas | f I : PwR SLR d(HR)d(LR) 
(58) 
- 70 
+ J f el] acur)a(tr) 


Using the expansion on eX for small x, eX = 1+x, Equation 58 becomes 








~(HR)* -(LR)* 
0 pw 20 20 
r(q) = 52 foc HR CER Py PCHRICLR) | acHeya(uR) 
280 uno Cunt 
HRLR JS, HRTLR 
=(uny’ =(uRy" 
Osa fe atumf eau) + 
HRLR J 0 
~(HR)? -(LR)? 
0 20 0 20 
ef The FR acon) f Ree ER a(ur) (59) 
-o SHR 0 LR 


Evaluating Equation 59 yields 








45 
(a) = 5 a (3 ‘ 2ro.q )(3 2r04q, ) 
0 o 
2 2 
~(LR -(in) 
2o1R 20H 
=a - VE 0 


r(q) = + & (-140)(0+1) 


rq) = 5-2 


* 
or 


p= 5-4 r(q) (60) 


Therefore from the relationship of Equation 60 it is shown that 


on aa 0 2 (61) 


2 
a ae Ey pe (s2) 
If one uses the two level or one-bit quantizer with input/outout 
relationship as shown in Figure 6(b) the variance in the estimate of o 
is the same as that given in Equation 62. Therefore, if the SSDA method 
is to be implemented with a one-bit quantizer, the choice of quantizer 


depends on which is easier to implement. 
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c. Resuits of S/N Ratio Analysis for Higher Order Quantizers 
The analysis of the degradation of the S/N ratio was extended for 
both the Direct and SSDA methods to the cases where three, four, and six 
level quantizers are used. As shown in Figures 6(c), 6(d), and 7, the 
variables v,/o, v)/o, Volos a and b are chosen such that a is minimized. 


Table 1 shows the results of this analysis for both the SSDA and Direct 


methods. The curves presented in Figure 12 show 


No Optimum 
Quantization Parameters 
Levels 


vo/a = 1.4 
a=3 





Table 1. Results of S/N ratio analysis. 


oH vs. no. of quantization levels for the two image matching methods 
being considered. Figure 12 clearly shows that the Direct Method yields 
a lower degradation in the ms S/N ratio for al} quantizers except the 
two level quantizer. Therefore, the conclusion can be made that the 


SSDA method would not be implemented using the higher order quantizers 


if the decision were based only on degradation of the ms S/N ratio. 
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However, since the SSDA method can be implemented with less and faster 
hardware than the Direct Method when more than two levels are used, it 


should not be ruled out as a candidate correlation algorithm. 
xt © - Direct Hethod 


& - SSDA Method 





No. of Quantization Levels 


Figure 12. Comparison of the SSDA and Direct Double Summation Methods 
From Table 1 and Figure 12 one can draw some conclusions about 
correlator design parameters (eg., reference array size, number of 
quantization levels, and statistical accuracy). From Table 1 the 
degradation in the mean square S/N ratio for both methods using any 
number of quantization levels is inversely proportional to the number 
of pixels used in the reference array (i.e., doubling the number of 
pixels in the reference array halves the S/N ratio degradation for a 
given number of quantization levels). Also from Figure 12 the degra- 
dation in the S/N ratio can be improved by increasing the number of 
quantization levels. However, relatively little improvement is gained 


by using more than four levels (2-bits) with either the Direct or SSDA 
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method white the hardware to implement more than four levels increases 
significantly. The conclusion is that a correlator using either the 


Direct or SSDA method should have the video pixels quantized to two, 


three, or four levels. 
If the correlator is to operate in real time, then the sampling 


period of the incoming live video dictates the amount of time available 
for each correlation computation. However, increasing the size of the 
reference array and increasing the number of quantization levels (eg., 
from two to four levels) both require more hardware and a longer compu- 
tation time. From Table 1 increasing the number of quantization levels 
from two to four increases the ms S/N ratio by 2.467/1.3 or 1.898 for 
the Direct Method and by 2.467/1.715 or 1.438 for the SSDA method. The 
computations, however, using four levels require more time. In fact from 
data books on present state-of-the-art hardware one could more than 
double the size of the reference array with this increased computation 
time using two level quantization. This increase in the size of the 
reference array would more than double the ms S/N ratio. Therefore, the 
trade-off is to make the reference array size as large as possible for 
real-time computation. Because of hardware considerations it is usually 
desirable to make each reference array dimension a power of two (i.e., 
36x16, 32x32, 32x64, 64x64, etc.). If for example a 32x32 reference 
array in a two level correlator can be computed in real time but a 


32x64 reference array can not, then one might consider using three or 


four levels in this 32x32 correlator if time would allow. In other 
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words the trade-off is to use as large a reference array as possible, 


then consider using more than two levels if time permits. 
One additional consideration when going from two level to three or 





four level quantization should be pointed out. When quantizing to two 
levels, each KxL subarray of the LR video should be quantized to one 
level when above the mean and to the other level when below the mean of 
that KxL subarray. For each value of p and q in Equation 3 or 6 all of 
the KxL pixels in that subarray should be quantized about the mean of 
that subarray. Quantizing about some moving average mean or about the 
mean of the entire NxM LR image leads to errors in the correlation 
computation. However, since it is not practical to do this, some trade- 
off technique is usually devised. If a three or four level quantizer 
is used, then the standard deviation as well as the mean of each Kx. 
subarray should be comprited. To use any other technique to find V,/6 
in Table 1 leads to errors in the correlation computation. In fact 
these additional errors could be larger than the gain in going from a 
two level to a three or four level quantizer. The amount of error 
induced by inaccurate values of mean and standard deviation in the 
quantization is scene dependent. The conclusion is that if one uses a 
three or four level quantizer, care should be taken in the computation 


of the mean and standard deviation in_ order to prevent a significant 


loss in the S/N ratio due to inaccuracies in these computations. 


6. Implementation of the SSDA and D*iect Methods Using One-Bit 
Quantization 


Wien both the HR and LR video are quantized to two levels, then 





from Table 1 the Direct and SSDA methods have the same ms S/N ratio. 
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In this section the implementation of the two methods using exclusive 
OR gates is shown to be equivalent. The conclusion is that when 
quantizing to two levels, the SSDA and Direct methods are equivatent. 
This is not the case when one or both video signals are quantized to 


more than two levels. In this case the SSDA method is not a true 


correlation technique while the Direct method is. 


a. Direct Method 

Consider first that the HR and LR video have zero mean and that 
the HR video is quantized by the input/output relationship shown in 
Figure 9. Then if the LR video is quantized by the input/output 
relationship shown in Figure 13, the correlator inputs when the input 
video match are given by 01 and 10 and video mismatch are given by 00 
and 11 input combinations. If the output of the correlator is to be 
1 when the input video match and 0 when the input video mismatch, then 
the input/output table is given in Figure 14 and can be implemented 
using an exclusive OR (@)) gate as shown in Figure 15. Implementation 
of the entire array is given in Chapter 4. Using this method, the peak 


value corresponds to registration of the two images. 


Figure 13. Reversed polarity one-bit quantizer. 
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Figure 14. Input/output table for one-bit correlation. 


HRg =) > 

LR 
Correlation 

value 


ADDER 


Figure 15. Exclusive-OR implementation. 


b. SSDA 

The SSDA implementation is similar to that given above, however the 
quantizer given in Figure 9 is used to quantize both signals. The out- 
cone of the SSDA algorithm is shown in Figure 11. The states 00 and 11 
now represent matched samples and 0] and 10 represent mismatch. The 
circuit for implementing the SSDA algorithm is the same given in Figure 15. 
The adder output is now an error value instead of a correlation value. 
Therefore the minimum value of the adder output corresponds to image 
registration. 

In this section it has been shown that if the two images are 


quantized to two jevels (one-bit quantization), ther the Direct Method 
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and SSDA methods are identical. In fact, either type correlator design 
can be converted to the other by reversing the polarity of one of the 
signals before quantization and looking for a minimum if originally 


Yooking for a maximum at the output of the correlator (or vice-versa). 





3. CORRELATION SYSTEM CONSIDERATIONS 


The various trade-offs and other considerations which lead to the 


correlation implementation in Chapter 4 are presented in this Chapter. 


A. Determination of Sampling Frequency 

Shannon's sempling theorem says that theoretically all the infor- 
mation in a signal is contained in the sampled signal (and can therefore 
be recovered from the sampled signal) if the sampling rate is at least 
twice the bandwidth of the signal (B). This sampling frequency te = 2B 
is usually referred to as the Nyquist rate. 

Consider a 30 frame per second 525 line TV system with 4:3 aspect 
ratio. Tne number of active horizontal lines is approximately 480. If 
the horizontal and vertical resolution of each pixel are to be the same, 
then (4/3)480 = 640 samples should be taken on each TV line. The sampling 
rate is (4/3)(480)(&25)(30) = 10.08 MHz or approximately 10 MHz. The 
sampling period for a 10 MHz sampler is 100 nsec. Therefore in order 
to design a real-time correlator each computation must be accomplished 
in less than 100 nsec. In other words the value of R for each p and q 
in Equation 3 must be computed in less than 100 nsec. However, the 
actual bandwidth for most 525 line black/white video signals is usually 
on the order of 4 MHz or slightly larger. The corresponding Nyquist 
rate is approximately 8 MHz. Since it is necessary to sync the sampler 


clock frequency to the video signal, the actual sampling rate will be 


Slightly less than 8 MHz as explained in Chapter 4. 
53 
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It should be pointed out that the recommended sampling rate of 
acproximately 8 MHz is based entirely on system bandwidth considerations. 
Since sampling at a higher rate will yield relatively little gain while 
decreasing the available time for real time correlation, 8 MHz is seen 
as an upper limit on the sampling frequency. If the sampling rate is 
decreased in order to allow more time for real time correlation or for 
shift register hardware considerations as explained in Chapter 4, then 
what one gives up is horizontal pixel resolution. The decreased sampling 
rate does not effect the vertical pixel resolution since this is deter- 
mined by the number of TV lines. Initial reaction to this would be to 
conclude that not much would be lost in correlation since as reported 
in [4] military targets such as tanks, personnel carriers, jeeps, etc., 
exhibit a larger spatial frequency content in the vertical direction 


than in the horizontal direction (i.e., there is more high frequency 


a RETTIG, SLES, aoe a 


spatial content in the vertical direction than in the horizontal direttion | 
for vehicle targets). A much closer analysis given in the following | 
section reveals that if the reference array is 32 x 64, most of the 

correlation is being performed on the target background and not on the 

target itself. Reference [4] points out that most backgrounds have 

greater horizontal spatial frequency content than vertical content. An. 
obvious example of this would be forests. The effect on the accuracy of 
correlation if one decreases the sampling rate from approximately 8 MHz 


to say 5 MHz is scene dependent and was not studied under this contract. 


It is recommended, therefore, that the sampling rate effects on correlation 


be studied in any correlator technology program. 
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B. Area Correlator 


Consider a 525 line TV system with approximately 480 active lines 
and a 2° vertical FOV. A standard 2.3 x 2.3 meter target would subtend 
6.3 lines in the TV FOV at 5 KM, 7.9 lines at 4 KM, 10.5 lines at 3 KM, 
and 15.8 lines at 2 KM. If the reference array is 64 x 64 (ina 
full-frame correlator), then the target occupies 0.967% of the refer- 
ence array area at 5 KM, 1.52% at 4 KM, 2.69% at 3KM, and 6.092% 


at 2 KM. Therefore, it is evident that most of the correlation is being 
performed on the target background and not on the target itself. If 


the size of the reference array is 32 x 32, then the above target area 
percentages would increase by a factor of four. 

The results of the above paragraph can be stated in a slightly 
different way. If two TV lines are required to display a spatial cycie, 
then the highest vertical spatial frequency detectable at 5 KM is 1.34 
cycles/meter, at 4 KM is 1.71 cycles/meter, at 3 KM is 2.28 cycles/meter, 
and at 2 KM is 3.42 cycles/meter. If the horizontal and vertical reso- 
lutions are the same (i.e., sampling at 10 MHz) then the limiting horizontal 
spatial ficy:ency resolution is the same as the vertical. If the sampiing 
rate is reduced to 7.5 MHz, then the limiting horizontal spatial 
frequencies detectable are 1.005, 1.28, 1.71, and 2.57 cycles/meter 
respectively at the above ranges. Stated another way, trees spaced 
every 0.75, 0.58, 0.44, and 0.29 meters apart respectively at the 5, 4, 3, 2 
KM ranges could be detected when sampling at 10 MHz while trees spaced 
every 0.995, 6.78, 0.585, and 0.39 meters apart respectively could be 
detected when sampling at 7.5 MHz. The above results are summarized in 


Tatle 2. 
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C. Preprocessing of Input Video 
1. High Resolution Image Preprocessing 
The purpose of preprocessing the HR video is to reduce the spatial 

resolution of the HR video to that of the LR video, a condition necessary 
for correlation purposes. This difference in resolution is caused by 
the differing fields of view (FOV), number of TV lines per frame, frame 
rate, aspect ratio, and sampling rate of the two TV systems. Due to 
size, weight, and cost constraints the FOV of the missile seeker will 
be W times (W>1) that of the FOV of the fine pointing and tracking 
system. If Wis an integer, then the preprocessor averages the first 
W columns of the first W rows of the HR image to obtain the P(1,1) 
picture element (pixel) of the reference image. Then the next W columns 
of the first W rows are averaged to obtain the P(1,2) pixel of the 
reference array as shown in Figure 16. If the sampled 'R video forms 
an A x B array, then the reference array is an (A/W) x (B/W) array. The 
size of a possible target in the reference array is then identical to 
the size of the target in the LR array. Equatior 63 is the preprocessing 
algorithm where W is an integer. 

Lor Asw 


Ww W 
P(I,J) = 4 > HR [ (va - WM), (W - wn) ; (63) 
W Mel N=1 1 B/W 


iA 
tA 


The assumptions made in deriving Equation (63) are that both TV systems 
have the same frame rate, same number of TV lines per frame, saine aspect 
ratio, are sampled at the same rate, and have linear optics. Figure 17 
illustrates in block diagram form the implementation of Equation 63. 


The hardware implementation is presented and discussed in Chapter 4. 
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Figure 17. 
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Preprocessing of HR image where W = 3. 
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If Wis not an integer, the preprocessing algorithm is more complex. 
Making the same assumptions about the two TV systems as made previously 
and defining 


U = smallest integer < (I - 1)W+1 


V = smallest integer S (3 -1)W4+1 
X = smallest integer Sw 
Y = smallest integer Sow 


the preprocessing algorithm becomes: 


i{ x Y 
P(IW) => f y _-HR(M,N) 
wo PMeU+L = N=V41 


+ oa If z (1-1)4] HR(U,N) + [tw a | cae. 


+ a. [v - (1-1) HR(M,y) + [2w a HR(M,Y+1) 


7m (u . (1-1)4 v a (3-1)4] HR(U,V) + [ow - | HR(U ,Y+1) 


+ [IW - xX] ' - (3-1) 4} HR(X+1,V) + [ow - ‘| HR(X+1,Y+1) ( 64) 


Figure 18 illustrates the pixel averaging where W = 3.25. The algorithm in 


Equation 64 can be checxed by looking at the P(2,2) element in Figure 18. 
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Figure 18. Preprocessing where W = 3.25. 


The possibility that the last column and last row of P(1I,J) elements 
turn out to have less than W x W elements of the HR array is of no concern 
since only the 32 x 32 or 32x 64 P(I,J) elements centered about the 
desired target will be used in the correlation algorithm. The important 
factor is to have the spatial resolution of the preprocessed HR array 
identical to the spatial resolution of the LR array. 

In Equation 64, W is the scale factor by which the resolution of the 
fine pointing and tracking system TV image must be reduced. The scale 
factor is due only to differing FOV and is the same in both the horizontal 
and vertical directions because of the assumptions made about the two TV 
systems. The effect on Equation 64 of removing tiiese restrictions will now 
be investigated. 


a. Number of TV lines per frame. 


If the number of TV lines in the HR image is twice the number of 
lines in the LR image and al} other factors are the same including the 
FOV, then the vertica! resolution of the HR image is twice the LR image 
and the horizontal resolution is half. The reduction of horizonai 


resolution is caused by the fact that the image beam is scanning twice 


long when sampling at the same rate. This could be corrected by sampling 
the HR image at twice the rate of the LR image. 

Let Wi, refer to the horizontal scale factor and Wy refer to the 
vertical scale factor between the HR and LR systems. Then if the ratio 
of the number of TV lines in the LR to the HR images is W, and al] other 
parameters are identical, the centribution of the different number of 


lines to Wi, is Wy and to Wy is V/M. 
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as fast in the HR image which causcs each pixel width to be twice as 
b. Aspect Ratio 
If the LR image has a 4:3 aspect ratio and the HR image has a 1:1 
aspect ratio and all the other parameters are identical, then the vertical | 
spatial resolutions of the two sampled images are the same but the hori- 
zontal spatial resolution of the HR image is 4/3 that of the LR image. 
The reason for this is that with linear optics the LR horizontal line is 
looking at a total scene which is 4/3 wider than the HR scene. However 


since the scan rate and sampling rates are the same, then each pixel in 


the LR image covers 4/3 as much horizontal target crea as the HR image. 


Therefore the horizontal spatial resolution of the HR image is 4/3 
greater than the LR image. 

In general if the aspect ratios of the LR and HR images are P, and 
P, respectively where P,/P. = Wo» then the contribution of the differing 


aspect ratios to Wi is Wo and to Ky is I. 


c. Frame rate 
I¥ the frame rate of the LR image is 30 frames/sec and of the HR 
image is 15 frames/sec and all other parameters are equal, then the 


horizontal spatial resolution of the HR image is twice that of the LR 
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image and the vertical spatial resolutions are the same. This is caused 
by the fact that the beam is scanning twice as fast in the LR system 
thereby causing the spatial width of each pixel to be double that of 
the HR image when the sampling rates are identical. 
In general, if the ratio of the frame rates of the LR to HR system 
is W, and al] other parameters are identical, then the contribution of 


the differing frame rates to Wi is W3 and to Wy is 1. 


d. Sampling Rate 

By an analysis similiar to that given in (a) - (c) above, if the 
ratio of the sampling rates of the HR to LR TV images is Wy and all 
other parameters are identical, then the contribution of the different 


sampling rates to Wi is W, and to Wy is 1. 


e. Field of View 
If the ratio of the LR to HR fields of view is Wie and al? other 
parameters are identical. then the contribution of the different 


FOV's to Wi is We and to Wy is We. 


f. General Spatial Resolution Matching Equation 
If all of the parameters in (a) through (e) above are considered 
at once, the total horizontal and vertical spatial resolution scale 


factors are given by 


= Wy Wy W3 Wy We 


=x 
I 


W, = Ne/W 
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U = smallest integer : (1-1)W, +1 
V = smallest integer < WI), +2 : 
z 166) 
X = smallest integer - Wy 
Y = smailest integer : JW, 
the preprocessing algorithm given in Equation 64 can be placed in the 


t 
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63 
Defining 
{ 


more general form given below. 


P(I,d) = gos at HR(M,N) 
whe a 


xX 
eo {tu - (1-1)W)] HR(U,N) + (IW) - x] un(as1 | 
N=Vel 


Y 
+ ans [tv - (J-1)W,) HR(M,V) + Cow, - Y) Hn(4, 2] | 
| 
+ (U- (1-1)W) [tv - (J-1)W,] HR(U,V) + COW, - Y] HR(U,Ye 2) | 
4 
| 
| 


+ [IW - x] [tv - (J-1)Wy] HR(X+1,V) + COW, - YJ waCxe1 Ya] 


(67) 
2. Low Resolution Image Preprocessing 
Since the mean value used for quantizing each Kxl. sub-array in the 
LR image is different, it is felt that a1] of the preprocessing can best 
be handled digitally. This section presents the optimal and sub-optimal 


techniques used to find the mean value and to quantize the pixels of each 


KxL sub-array for the two level quantization case. 
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Optimally for each value of p and q. in Equation 3 and “igure 4 the 
K x L pixels should be quantized about the mean video level of that sub- 
array. This coutd be done by storing K lines of the low resolution video 
in say 4 or 5 bit memory, computing the average value of each K x L 
sub-array as each new pixel is added, then quantizing all K x L pixels 
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about the mean for each value of p andq. It is feit that this method 
is far too time and power consuming for the amount of accuracy gained. 


Several sub-optimal trade-offs will now be presented. 
Since the mean video level is not expected to chanye appreciably 


from frame to frame, one method of reducing the real-time computational 
burden is to use the mean video level of each sub-array computed from 
the previous frame. This method, however, requires additional memory 
and additional host computer load. 

A second possibility would be to quantize about the same mean for each 
sub-array until the mean changes by more than some precomputed amount at 
which time that entire K x L sub-array would be requantized about the new 
mean. This method, however, still requires the hardware to requantize an 
entire K x L sub<array. 

A third possiblity is to store the previous K lines of digitized 
video in 4 or 5 bit memory, compute the average value of each K x L 
sub-array as a new pixel is digitized, and requantized only the last coiumn 
of the array about the new mean. This would require only K comparators 
rather than KL comparators. The average can be computed by storing the 
sum of each column in shift registers. When a new pixel is added, the 
first column sum is subtracted from the total and the column containing 


the new pixel is added to the total. 
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A fourth possibility is to use the method of the previous paragraph 
but quantize only the added pixel about the new mean leaving al] other 
quantized values the same. This would requir2 only one comparator. Using 
this method only the lower right pixel in the K x L array would be quantized 


exactly as shown in the solid area of Figure 19. If the mean video is 
assumed to vary linearly with distance from this pixel, then the upver 


left pixel has the largest error for this value of p and q. The fifth and more 
accurate method on the average would be to compute the quantization level 

of the marked pixel by computing the mean level of the dashed K x L array 

as shown in Figure 19. This would mean a delay in each correlation 


computation of approximately K/2 lines plus L/2 sample intervals. 
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Figure 19. Method for pixel quantization. 


The sixth possibility is to quantize about the average of the HR 
reference. It is felt that this method, while being by far the easiest 
to immlement, sould lead to too many false registration points and would 
be unacceptable. 

A further study involving typical military scenes is needed to 
determine which of the above trade-offs is the best to implement. Without 


the benefit of this study, method Five is recommended and an implementation 
is suggested in Chapter 4 of this report. 
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3. Effect of Quantization about Incorrect Mean 


As pointed out in the previous section, one of the sources 01 error 
in the two level by two level (one-bit by one-bit) correlator is quanti- 
zation about an incorrect n:an. The effect on the mean square signal/ 
noise ratio of quantizing an entire array about the same incorrect mean 
is presented in this section. The effect on the ms S/N ratio of 
quantizing each pixel about differing incorrect means is not presented. 
However, the results can be used as an upper bound if one uses the 
maximum expected mean error in the resulting curves. 

Consider the HR and LR images quantized respectively with the 
quantizers shown in Figure 20. These quantizers represent an error in 
quantizing around the wrong dc level of the HR and LR images. Although 
the above quantizers assume that the HR and LR signals are zero mean, 
this does not necessarily have to be true. For HR and LR signals that 


are not zero mean the dc error due to quantization would be given by 


AX, = X,-HR and AX, = Xo~LR. 





Figure 20. Quantizers for the HR and LR images. 
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a. Direct Method 


Consider the first line of video 


L 
R(1,q) = } HR(1,m) LR{,m+q) (68) 
m1 


L 
M 


Dropping the row 1 notation and quantizing the HR and LR signals as shown 
in Figure 20 yields 


A L 
R(q) =; Ly (HR(m)] g, Rime) } (69) 
m= 


The variance in the estimate of R is 
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On = <R2> - <p>? (70) 
R 
where 
A 1 L 
<R> = E{= } g,(HR(m)] g.[LR(m+q) ] (71) 
Te eek 2 
m=1 
and 
2 
2 L 
cR°>= EY 12 J gy) (HR(m)) g,(ZR(m+q)] (72) 
m=] © 
Assuming sample pair independence, a gausian distribution of the 
HR and LR signals, and evaluating the resulting expectation integrais 
yields 
x x 
go Se }. - 16 erf* (—) erf? 2 (73) 
R L 0 Oo 
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Using Price's Theorem to evaluate r(t) where p is the correlation 


coefficient between HR and LR yields 


9g, 9g 
dr _ 2 1 2 (74) 
ap < (oR) (srR)? 


Evaluating the partials in Equation 74 by inspection of Figure 20, 


assuming a jointly gaussian distribution for the HR and LR signals and 


evaluating yields the following relationship between 0," and 7. 
2 2 
xy Xo 
ae 2 
2 mT, 2 oO 0 2 
oy = (5) e e ue (75) 


Substitution of Equation 73 into 75 yields the desired effect of the 


incorrect mean on the ms S/N ratio. 
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(76) 
b. SSDA Method 


The registration function for the SSDA method is given by 


L 


l 
R(q) = ; 1 |HR(m) - LR(m+q) | (77) 


Going thru an analysis similiar to that used for the Direct Method 
yields the identical result for 0 given in Equation 76. This was 
expected, however, since the two methods have been shown to be equiva- 


lent in the one-~bit by one-bit case. 
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Figure 21 shows a parametric set of curves depicting the change in ; 
0, for various values of (J) and ase The dotted curve in Figure 21 
shows the effect on 0,” if the dc errors in the quantizers are equal. 


If both x and Xo are zero (i.e., no error in the mean), then 
2 
p 

from Figure 21 the variance is essentially insensitive to small dec 


x L = 2.467 which is the result obtained in Table 1. As can be seen 


errors but grows very rapidly for errors in the mean above 0.40. 





X,/o 


Figure 2]. Effect of dc quantization level on the variance in the 
estimate of po 
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4. Preprocessing for Differing Sensor Characteristics 

When trying to accomplish target hand-off between two systems. it 
is very important for the two sensors to have similiar characteristics. 
This of course is the reason for the necessity for the algorithm which 
equalizes the spatial resolution of the two images as given in Section 
C-1 of this chapter. It is also very important for the spectral charac- 
teristics of the two sensors to be the same since correlation can be 
accomplished only if the two video signals are almost the same. If one 
sensor covers a wider frequency spectrum than the other, an optical 
filter might possibly be used during correlation to cause the two 
spectral responses to be the same. If the two sensors have completely 
different spectral responses, such aS one sensitive in the visible 
spectrum and one in the infrared spectrum, then correlation would be 
difficult to accomplish due to the difference in signal characteristics. 
However one possibility in this case would be to preprocess the video 
such that only the edges are used for correlation. This is a possibility 
since even if there is a cont-ast reversal between the two signals, the 
edges in the FOV should be approximately the same. 

Preprocessing of the HR and LR arrays concerns changing the form 
of the data so that a better registration between the two can be 
obtained. In most of the literature that has been surveyed this task 
is referred to as outlining, edge detection, or boundary mapping. These 
methods are used mainly for cases in which the two images are acquired 
at different times sr the images come from different types of sensors. 


The first of these reasons is not applicable to the present system under 
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Study. It is quite possible that for the system under study the images 
could be produced from different types of sensors. In this situation 
the data representation of the ground scene from the two sensors might 
be different. Then some type of preprocessing of the data, such as 
boundary mapping could be employed. 

Reference [5] describes a method by which boundary maps are pro- 
duced from the raw data. Essentialiy this method calculates the 
differential change in magnitude from one data point to the next. It 
does this for both X and Y directions away from the point being examined. 
The algorithm then decides if the point being examined represents the 
boundary of a region or not. If it is a boundary, then that location is 
assigned a value of 1. If it is not a boundary point, a 0 is assigned 
to the location. Once a boundary map is produced for the entire image, 
a correlation can be performed using boundaries of objects in the image 
instead of all the raw data from the image. 

If the data received from the HR and LR imaging systems are of 
different forms (such as would occur with different sensor characteris- 
tics) then the method of boundary maps or edge detection would warrant 
further investigation. Implementation of any of these methods, however, 


requires additional hardware and computation time. 


D. Vibration Analysis 
The effect of LOS jitter of the PTS and missile seeker systems on 
the correlation algorithm accuracy is investigated in this section. From 


talks with MICOM engineers the jitter of the seeker LOS should be less 
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than 100 urad rms and is the value used in this section. The LOS jitter 
of the PTS system has been demonstrated to be less than 30 urad for at 
Teast one system. These values are the total rms jitter due primarily 
to disturbances from the helicopter at 11, 22, 33, and 44 Hz. 

The primary effect of LOS jitter on the correlation algorithm is 
to cause a distortion in the stored video scene. For example. if the 
jitter is causing the LOS to move downward during the sampling and 
storage of a 32 x 64 element video array, then the 32nd line may actually 
be looking at the scene the equivalent of several TV lines below where 
it should be thus causing a stretching distortion. If the LOS is moving 
up due to jitter, then the 32nd line may actually be looking at the 
scene the equivalent cf several TV lines above where it should be thus 
causing a compressing distortion. The problem is further complicated 
by the fact that one video image may be compressed and the cther stretched 
by the jitter. The net effect on the correlation algorithm is to cause 
a smaller correlation peak which can be modeled by a reduction in the 


Sianal-to-noise ratio 





In the calculations which follow both TV systems are considered to 
be 525 line systems with 480 active TV lines (the remaining 45 lines are 
used in the vertical retrace), and to have 60 half-frames per second or 
30 full frames per second (2:1 interlace) with a 4/3 aspect ratio. For 
a sampling rate of 7.5 MHz the low resolution image will be digitized 
into a 240 x 480 element array during the first half-frame and 240 x 480 
element array ducing the second haif-frame. If the correlator is designed 


such that correlation values are calculated as data comes in, then correla- 


tion is being performed during each half-frame (1.e., the 32 rows used in the 
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correlator all come from the same half-frame). Therefore, the total 
time elapsed from the first sample in each 32 x 64 array used in the 
correlation algorithm to the last will be the time required to sample 
31 complete lines plus 64 samples of the following line. 


Tp” 31_+ (64/480) . 9.00197 seconds (78) 
525 x 30 


As stated previously the LOS jitter is predominately at 11, 22, 
33, and 44 Hz. If one assumes that the 100 urad rms jitter fs all at 
44 Hz, the worst possible condition for correlation, then the greatest 
angular disturbance which the LOS would have in 0.00197 seconds would 
be twice the value of the 44 Hz sinewave evaluated at t = .00197/2 


seconds. 


donax 2(141.4) sin{2r44(0.000986) J 


or 


Ao 76.16 urad (79) 


max 


If the seeker vertical FOV is 2°, then each TV line is scanning 72.7 
urad if one assumes 480 active TV lines. Therefore the maximum movement 
of the LOS from the first to the last sample taken for each computation 
of R would be slightly over one line. Since correlation in the half- 
frame correlator is being performed by using alternate lines of the 

full frame, it is felt that this amount of movement will not cause a 
Significant problem. A more detailed analysis could be obtained by 


modeling this LOS movement with an additive noise source which would 


degrade the S/N ratio. 


a ek ee “ Sao ie 
NAO 9 int ERR PCO me Ps NY LRG AE Vice cai eae py 





aM abravele ar eit. seit Lad beg rece 2 er el ea tae tte: 


fete. atten ans thaadbes ALD ts bi Sommaire tae AO eae EE 





74 


Next consider the case where correlation is performed using 16 lines 
from one half-frame and 16 lines from the succeeding half-frame. The 
32 x (4 correlator then operates on 64 samples from each of these lines. 
The total time required to sample the data used for each computation of 
R is 

Tp = 2 A/N80 + 1s 0.0176 seconds (80) 

525 - 30 60 

which is more than half a cycle of a 44 Hz waveform. Therefore, the 
total movement could be as much as the peak-to-peak LOS jitter or 282 
yrad. This represents approximately 3.89 TV lines. Even if all of the 
jitter is at 11 Hz, the maximum angular movement would be 162 urad which 
is over 2 TV lines. Using the full-frame correlator the correlaticn 
peak would be reduced more than for the half-frame correlator. The 
net effect of the LOS jitter is a laracr reduction in the S/N ratio for 
a full-frame correlator tran for a half-frame correlator. 

Table 3 shows the maximum possible angular vibration in prad and 
its equivalent in TV lines if a 100 urad rms jitter in the seeker LOS 
is considered to be concentrated st i], 22, 33, or 44 Hz. The data for 
a 32 x 64 element correlator operating on each half-frame respectively 
and for the same correlator operating on a full-frame are given. From 
the data in Table 3 it is obvious that a 32 x 64 correlator operating 
on successivie half-frames is much less sensitive to LOS jitter than a 
full-frame correlator. The half-frame correlator is also simpler to 


implement since data does not have to be stored from one half-frame to 


the next. 
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Table 3. Angular disturbances in seeker LOS due to jitter. 





Next consider a PTS system with a vertical FOV of 0.4°. With this 
system five lines of video must be averaged in order to equalize the 
size of the target in the two FOVs. Therefore the elapsed time between 


first and last sample will be 


5_x_ 32-1) + (64/480) = 9.01 seconds (81) 
525 + 30 


If one assumes that the 30 urad of jitter is all at 44 Hz, then the 
largest angular disturbance which the LOS would have in .01 seconds 


would be 


Mbnax = 2(72) (30)sin{2n44(.005)] 


or (82) 
83.35 urad 


u 


Ao max 
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For a vertical FOV of 0.4°, each TV line, assuming 480 active lines, 
scans 14.5 urad. Since each five successive lines of the PTS video are 
averaged, the b$nax is slightly more than one TV line when averaged. 
The net effect of LOS jitter on the PTS is therefore seen to be 


approximately the same as the effect on the seeker system. 


E. Derivation of aX and AY Misalignment in LR Image 

After the correlation array, R(p,q), has been calculated, it is 
desired to move the missile seeker LOS such that the target is in the 
center of the LR FOV. In order to do this the misalignment in both the 
X and Y directions must be determined. The position of the target in 
the LR image can be determined by performing a search of the R(p,q) 
correlation array for a maximum in order to identify the registration 
points p* and q*. 

The geometrical layout of the HR and LR images is shown in Figure 
22. The p and q counters start in the upper left corner of the LR 
image. This designates the starting placement of the HR image in the 
LR image for the calculation of the R(p,q) array. 

Figure 22 also shows a possible location of the HR Image which 
produced a maximum value of R(p,q) over all possible p and q. It is 
desired that image registration take place in the center of the LR 
image. The K x L array of LR image points around the center is shown 
in Figure 23 along with the geometric definitions of AX and AY. The 


displacements AX and AY represent values by which the LR camera must be 
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(F.q) = (0,0) ,Starting placement y 







HR Image for 
maximum in 
R(p,q) 


Figure 22. Image layout identifying p* and q*. 


Center of 
LR Image 


ake 


t AY 





Figure 23. Image layout showing AX and AY. 
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shifted so that the maximum value of R(p,q) will occur at the center of 


the LR image. It is desired that 
p* = N - K (83) 
and 


> (84) 


9) es 
Using this reasoning AX and AY can be calculated using the following 
equations: 


AX = qt - (ee) (85) 


and 


AY (E) - p*. (86) 


The AY and AY in Equations 85 and 86 are expressed in terms of pixel 
elements. The angular errors Ady and Ady in a seeker with a FOV of 2°, 


480 active TV lines in this FOV and 480 samples per line are 


AX 


doy = Za * 2°, in degrees : (87) 
doy = bY x 2°) in degrees (88) 
Y 480 


These angular errors can be used to torque the seeker gimbals such 


that the target is in the center of the FOV. 
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F. Threshold Effects on Probability of False 
Registration and Detection Probability 


The following analysis develops the concepts of the probability of 


false registration (i.e., the probability that two images are chosen as 


registered when they in fact are not) and the probability of detection 
(i.e., the probability that two images are chosen to be registered 
given that they are in fact registered). Each of these probabilities 
is given in terms of the tnreshold for deciding if registration is 
achieved and the input SNR for both images. 
For simplicity and because other considerations «ndicate its 
desirability, the following is developed fcr a 1-bit by 1-bit (2-level | 
by ?-level) ‘correlator using either the Direct Double Summation Metinud 
or the Sequential Similarity Detection Algorithm. Previous comparisons 


have shown these two methods to be identical statistically for the 1-bit 


quantization case. 
The ,:ollowing assumptions are inherent in the deve?opment. 
1. Signal and noise levels at each pixel are independent of each 


other and of signal and noise levels from any other pixel. 





2. The noise at any pixel is zero mean Gaussian for both the HR 





Sr ee eee 
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and LR images. 
3. the rms noise level 1s constant (o,, ) for each pixel of the 
HR 
HR image and a different constant (o, ) for each pixel of the LR image. 


ER 
4. The signal levels for the HR and LR images have mean values 


HR and LR respectively for the pixels within each image. It is of course 
realized that LR may change ac p aad q (the shift values) change; however 


for a given correlation computation ER is a constant. 
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5. The signal plus additive noise from HR and LR images are 
subjected to 1-bit quantization as represented by Gy and Io in Figures 
24 and 25. 

The only possible outputs for a one-bit implementation are zero or 
one for comparison of a LR pixel with a HR pixel. Therefore, for 
implementation of a K by L. array correlation, the correlation magnitude 
takes on values ranging from 0 to KL in integer steps. The ovvious 
questions which arise and the answer of which is the purpose of this 
section is: "How close to the maximum value of KL must a given corre- 
lation be to indicate registration? What are the consequences of 
chosing a particular threshold for a decision on registration of the 
Hk and LR images?" 

Under the assumptions outlined above we can write the probability 
density functions for the noise in any pixel of the HR and LR images 


for a given shift value (p,q). 





2 
“THR 
20, & 
] HR 
f (n ) =: e (89) 
ig Me N20 
2 
“LR 
20, 2 
f (np) Sp Be led oe e LK (90) 
MR V2x0,, 


At pixel (n,m) in the HR image and pixel (n+p, m+q) in the LR image we 


have signal plus noise as follows: 


2 z : 91) 
up = Pup (nam) HR(n,m) + Nip (nom) ( 
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rR = HR(n,m) + Mpinsm) 


4 94 (Typ) 
cere 
rrr? LR(n+p,m+q)+n, _(n+p,m+q) 
9 92(r/p) 
input = signal + noise quantized output 


Figure 24. Quantized signal plus noise. 








Figure 25. Quantizer characteristics. 
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Mea = “pint. mq) = LR(ntp,mtq) + n, p(ntp, mtq) (92) 


where Yup and rip are to be quantized and compared logically and the 


output summed for Ki pixels as shown in Figure 26. 








(1 if match) 


0 if no 


OUTPUT 
(0 to KL) 


Figure 26. Correlation technique. 


From the quantizers 9) and J we have the rules for no noise 


9, = 1 if HR > HR 
_ (93) 
= (0 if HR < HR 
go = (lif LR>TR 
5 =e (94) 
= {O if LR < LR 
If noise is included, the rules become with noise 
g, = (1 if HR+tn,, > HR 
1 | HR (95) 
= (0 if HR+Nip < HR 
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Clearly there are four hypotheses that can be postulated for the 
outputs of the two quantizers in terms of the inputs. The 


hypotheses are: 


Hupg ~ the input HR < HR (choose 9,70) 
Huy 7 the input HR > HR (choose 9,71) 
Hag 7 the input LR < LR (choose 9570) (97) 
Hpi 7 the input LR > LR (choose 9571) 


These hypotheses can be chosen with no error by a simple threshold 
circuit if no noise is present. If noise is present, there are possi- 


bilities for error in the choices. This error is dependent on the signal 





levels, the noise statistics and the number of pixels used in a given 
corre;ation computation. 
The correlation output is then compared with a threshold value, No, 


to determine if the images are registered. If the correlation output 













is greater than No» the images are chosen to be registered. Since No 
is a parameter which must be chosen, its effect on expected errors must 
be known. Specifically its effect on false alarm and detecticn proba- 


bilities must be known. 


1, False Alarm 

False alarm occurs when the logic comparison indicates registration 
of the image with a particular LR image when in fact the two are different. 
The probability of false alarm is best examined by considering the 


probability of false alarm for each comparison of an HR and an LR pixel 
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pair. Defined for each pixel pair, the false alarm probability is 
simply the probabiiity that the HR and LR pixels are chosen matched 
given that they are different. However, since the HR and LR images 
can be different (not registered) without requiring that al] KL pixels 
be different between the two images, consideration must be given for 
the probability of matched pixel pairs within non-recistered images. 
This consideration requires an additional assumption about the proba- 
bilities of pixel pairs being matched or mis-matched given that the 
images are different. Fortunately, based on fundamental arguments 
supported by the method of quantizing about the mean value for each 
image, these assumed probabilities are easily obtained. 

Since the relation of total number of chosen pixel matches to the 
threshold No is really what determines false registration probability, 
it is necessary to examine the probability of cnoosing pixels as 
matched given that the images are different. Clearly this has two 
parts: 1. probability of [choosing pixels matched|pixels matched| 
images different], and 2. probability of [choosing pixels matched| 
pixels different |imaces different]. 

If the individual pixels for different images are assumed to be 
independent of each other, then a "0" or "1" is equally likety at any 
given quantization step. This should be true for any symmetric proba- 
bility density functions of the HR and LR intensities since quantization 
is about the mean value in each image. With this assumption, it can be 
shown that for each pixel pair comparison, the probability of false 


alarm is independent cf the image signal-to-.i0ise ratios. 
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The probability of false alarm for the ith pixel pair comparison 
is 
PCE] = [MoM p + (1-Min)(1-M) 9) ] (98) 
*P[pixels match|different images] 


+0(1-Mue Mp + Mup(1-M) p)]°PLpixels different| images different] 
where: 


wae e dp 


od ae 


M 


SNRHR = HR image signal-to-noise ratio 
SNRLR = LR image signal-to-noise ratio. 

Under the assumption stated above P[pixels match|images different] 
= P[pixels different|images different] = 5 and PCE] = 53 for all SNRHR 
and SNRLR. 

If all KL pixels are considered, we find that if N is the number 
of false alarms in KL pixels then the probability density function for 


N is given by 


PCE] = (SL | aN, (KL-N) : (99) 
N N 
where Pyle] is the probability of exactly N false alarms and obeys the 


familiar binomial probability Taw 
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where (") = (KL)! (100) 
N!(KL-N)! 


is the binomial coefficient and gives the number of possible combinations 


of KL pixels with N false alarms, and 


aw 
’ 


= P.[E] = probability of false alarm on each pixel pair 


1- P .fE). 


oa 
ua 


If a threshold value of N is chosen, i.e. N = No» such that if 
N < Np for any given correlation computation the decision is to reject 
the hypothesis that the images are alike, then a probability of false 


registration can be found as a function of No from 


PeqlNg] = Probability [N > No| images different] 


or 


N 
0 
r = KL N, - 
Prato! =]- } ( i) a (KL N) (101, 


For KL large, the binomial probability law is approximately gaussian 
which gives 
‘ 
No - (Kia + 4 
Pea(Ny) = 1 - M + (102) 
(KL )ab 
\ 


where 
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Probability of false alarm is plotted versus threshold, No» for 
KL = 2048 in Figure 27 and for KL = 1024 in Figure 28. 


2. Detection 


Detection or correct registration occurs when the HR and LR images 
are correctly chosen to be identical; that is, they are chosen matched 
given that they are matched. Similar to the development for false 
alarm, it is convenient to start with a single pixel pair comparison to 
derive the statistical behavior of correct registration. 

The major difference between the probabilities for false alarm and 
detection lies in the fact that when the HR and LR images are registered, 
the probability that pixel pairs match should be higher than it was for 
the unlike images examined in the false alarm treatment. Furthermore, 
this difference in probability of match and mismatch for each pixel 
pair results in the probability being dependent on signal-to-noise 
ratio's of the two image signals. 

The probability of choosing a pixel pair matched given that the 


HR and LR images are registered is 
PiLC] = (MM + (1 - Min htl - MQ] 
-P[pixels match| images same] 
+E) - Magi pn + Magi - Mad 


-P[pixels mismatch| images same] (103) 
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or if we define the error probability 


e 4 P[pixels mismatch| images same] 


then 


P.[C} = [MoM p + (1 - Mp ( = Mp] (1-e) 


+{(1 - MupnMip + Mig(1 - Mp] €. (104) 


For the ideal case, « = 0, each HR and LR pixel pair will match so 
that additive noise is the only error source. However, for practical 
sensors, there is a finite probability that pixe? pairs will mismatch 
due to different spectral responses, non-linearities and other error 
sources. Thus it is important to examine tne effects of non-zero 
values for c«. Since Min ana Me are dependent on signal-to-noise ratios 
SNRHR and SNRLR of the respective video signals, then as indicated by 
Equation 104 the probability of choosing a single pixel pair as being 
matched, given that the two images are alike, is dependent on the three 
parameters SNRHR, SNRLR and ec. 

Once the value for P.(C] is established then the total detection 
probability as a function of the threshold, No» can be determined. If 
all KL pixels are considered, we find as with the false alarm problem 


that if N is the number of pixels chosen as matched, then the probability 


density function for N is again binomial. 


PylC] = (4) oN gfKt-N) (105) 
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And the probability of detection is the probability that N will be 
greater than No» or 


PaLNyJ = Probability [N > No| images alike] 


N 
0 
Poly] = 1 - J (5) g{KL-N) (106) 
where 
c= P;{C] 
d=] - P.[C] 


Clearly, Py is a function of KL, Np and P.[C} which in turn depends on 
SNRHR, SNRLR and «. Figure 27 shows Py versus threshold, No» for 

KL = 2048 data points and for three worst case values of P£(C]. (NOTE: 
P.[C] is listed as PIC on all figures). 

A value of PIC = 0.5 gives exactly the same curve for Po as is 
shown for Peat This could occur only if we assume a value of c« = 0.5. 
This is highly unlikely since it says, that even thougn the images are 
alike, there is at best a 50-50 random chance that individual pixel 
pairs matc.1 between the two images. If the sensor characteristics are 
so different that « < 0.5 is possible, then there is no need to waste 
effort trying to correlate the two images. Figure 28 shows similar 
curves for Py with KL = 1024. 

As an example of what the curves in Figures 27 and 28 mean consider 
that it is desired that a high Po be attained while keeping the number 
of false detections to one per second. The number of false detections 


is given by 





4 
i 
i 
j 
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where: 
N. = number of shift values "p" for which correlation 
P is computed 
No = number of shift values "q" in the other direction 
F = frame rate (frames/sec) 
T,, = total mission time (1 sec) 
Nep = Pea + [640-64][480-32] - 30 


(107) 


—_ 
he 


6 
8(10 )Pen 


which requires Pea < 1.2(1077). Then from Figure 27 this Pea corresponds 
to a threshold of No * 1140. 

Projecting up to the Pp curve for PIC = 0.55 indicates a detection 
probability of less than 0.3 which is unacceptable. For PIC = 0.5 the 
value of Py is greater thai, 0.9999. Evaluation of this result says 
that if a false alarm rate of one per second is to be achieved then to 
maintain hiyh detection probability, SNRHR, SNRLR and < must be such 
that PIC > 0.55 and preferably > 0.6. 

The relationship between PD and PEA is shown more clearly in Figure 
29 where No is the common variable which has been removed. Improvement 
in performance is represented by movement of the curves diagonally 
upward from the coordinate center. 

Incidentally, a false alarm rate of one per second is the case 
where one false registration occurs out of ~3(10°) correlations. 


This is the number of real time correlations computed per second for a 
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94 
32 x 64 reference array and a 480 x 640 low resolution image at standard 
TV frame rates. 

Since the above results are so strongly dependent on PIC a parametric 
analysis of PIC is of value. 

The effect of « on the value of PIC is shown in Figure 30 for four 
discrete values of « as a function of signal-to-noise ratio. For 
convenience in plotting, only values corresponding to equa? SNR for the 
video signals are shown. The figure shows that c« has a profound effect 
on the maximum attainable value of PIC which is reached asymptotically 
as the signal-to-noise ratios increase. In each case, the maximum 
achievable value of PIC is I-c, and is achieved only as the two video 
SNR's increase. 

Since the HR image SNR and the LR image SNR will most likely vary 
independently of each other, it is of interest to examine the effect of 
different SNR's on PIC. Figure 31 illustrates this effect for a fixed 
value of c« = 0.1. As in Figure 30 the absolute maximum attainable PIC 
is l-c; however, it is not reached unless both video SNR's are greater 
than 3. The point made by Figure 31 is that for optimum detection 
probability, both images should have a SNR of at least three. Only 
partial compensation is achieved by one video signal having very high 
SNR if the other video signal has SNR significantly less than 3. 

To summarize briefly, the above analysis indicates that for 
signal-to-noise ratios which can reasonably be expected from electro- 
optic sensors (~3), even for a worst case assumption about sensor 


characteristic mismatch (c = 0.4), for the 32 x 64 reference array size 
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Figure 30. Effect of SNR and pixel mismatch probability, «, on 
probability of a correct decision for each pixel. 
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Figure 31. Effect of different video SNR on probability of 
making correct decision for each pixel. 
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a threshold can be set which will ensure low false alarm rate and high 
detectior probability. It should be stated that the above analysis is 
based on a one-look correlation for each shift value. Multiple-look 
correlation, that is, computing the correlation function for a given 
shift value several times and using a decision based on the multiple 
correlations, wil! provide even better performance statistically. How- 
ever, since multiple correlations in real time would require parallel 
processors, increased processing speed or reduced data array size, that 
technique can be eliminated for practical reasons. More than adequate 
statistical performance should be achievable with the one-look real-time 


correlation technique. 


G. Target Not in Seeker FOV 

Due to inaccurate boresight alignment or to movement of the missile 
within its stores rack, there exists the possibility that the target 
will not be within the seeker FOV. If the correlation coefficient does 
not exceed the threshold value during n consecutive TV fields, the 
decision is made to slew the seeker TV LOS in a search pattern to find 
the target. The value of n is a design parameter which should be 
studied in a technology program. A value of from one to four would be 
an adequate range for such a study. 

Figure 32 illustrates two possible search patterns. Figure 32(a) 
Shows a search pattern which searchs for the target over an area which 
is nine times the area of the original search area in eignt discrete 
steps of the seeker LOS. Figure 32{b) shows a search pattern which 


searchs for the target over an area which is four times that of the 
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Search patterns for missile seeker. 


Figure 32. 
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original area in four discrete steps. Search pattern 32(b) would 
require less than half the time as seerch pattern 32(a) if say the 
target is in the last possible dotted square. Due to recent experiments 


conducted at MICOM where the missile seeker was slaved to the PTS, 


it is felt that the search pattern of Figure 32(b) will be adequate 

i.e., the probability of finding the target in search pattern 32(b 

is quite high). However, in a technology program, tests should be 
conducted to determine the optimal search pattern (that is one that 
yields a high enough probability of finding the target in the least 
possible time). Another reason for choosing search pattern 32(b) is 
because of the inaccuracies of the correlation algorithm around the 
edges of the FOV. Since the edges of the original FOV in Figure 32(b) 
are well within the search pattern FOVs, the possibility of missing 
the target due to edge effect is reduced. If one uses the search 
pattern of Figure 32(a), there exists the possibility of not finding 
the target at all even though it is within the original FOV but close 
to an edge. 

The correlation values are ignored during the slewing of the seeker 

LOS. When the LOS stops, if after n TV fields the correlation does 
not exceed the chosen threshold, the LOS is advanced to the next search 
area. If after completely searching all areas the threshold is not 
exceeded, the gunner would be flashed a sijnal to this effect whereupon 


he can re-initiate the process if he desires. 














4. IMPLEMENTATION 


The system required to implement either of the correlation and 
location functions presented in Chapter 3 will consist primarily of 
hardware to perform the correlation operation itself. The speed of 
operation of the entire system w.:! be limited by speed considerations 
in the correlator. The correlation activity consists of a comparison 
activity (absolute difference, equivalence (@) or multiplication) 
followed by a summation operation. In order to speed the summation 
process, an adder tree is employed. The number of layers of addition 
and hence the speed of the adder tree is a function of the fan-in or 
number of branches on each layer of the tree as well as the propagation 
delay of the logic used to implement the tree. It is of major importance 
that maximum fan-in be employed whenever possible. Digital impltemen- 
tations treat addition as a binary operatior. Therefore, the number cr 
layers in the adder tree is directly related to Tog oN where N is the 
numoer of functions to be summed. Dramatic speed ircreases will be 
obtained by increasing the fan-in. A later section will illustrate 
this phenomenon. The method of presentation will be to discuss the 
various components of Figure 33 by first examining correlation techniques 


followed by discussion of the various support components. 


A. Implementation of One-Bit by One-Bit Real-Time Correlator 

In this section the logic design for a one-bit by one-bit corre- 
lator is considered. The objective is to find the largest reference 
array which can be handied in real time subject to a variety of con- 


straints such as logic family, power requirements, etc. If has been 
101 
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shown that each value in the R correlation matrix must be found in about 
133 nsec when sampling the incoming video at 7.5 MHz. Two logic families 
will be investigated: ECL and TIL Schottky. No slower logic families 

can meet the computational speed requirement for even the smal'est 
reference array. The technique employed is to use an exclusive-OR or an 
exclusive-NOR circuit to perform difference or multiplication respectively 
and then to sum these correlation elements with an adder tree. 

Figure 34 is the logic diagram for a 12 by 1 correlator. One can 
think of this #s one column in an array correlator. The inputs to the 
exclusive-OR gates are quantized one-bit signals. Figure 35 shows how 
16 of these correlators can be combined to yield a 12 x 16 element 
correlator. The circuit type, packaging, power, and propagation times 
are given in the Figures below the appropriate blocks. Al] MxL 
correlators can be designed in the same way. The 12 x 16 correlator 
was chosen for illustrational purposes. 

Table 4 is a summary of total packaging and power requirements, 
and total propagation delay for several reference array sizes using ECL 
and TIL logic. The largest real-time correlator using ECL logic is 
seen to be 48 x 64. The largest array using TTL logic is less than 
12 x 16 aid is not shown since this is unacceptable. One can observe 
from Table 4 that the ECL logic is about twice as fast as the TTL logic 
for a given array size. However, the total power requirement is ahout 
twice as great for the ECL logic as for the TTL logic. Since the 
number of packages is about the same, the physical size of the corre- 
lators should be about the same unless heat sinks are necessary for the 


ECL correlator. 
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Figure 34. Logic for 12 x ] correlator 
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Figure 35. Extension of Figure 34 to produce 
a 12 x 16 correlator. 
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Table 4. Comparison of ECL vs, TTL logic correlators. 


It is possibie to extend the hardware in a simple way to enconipass 
larger arrays with little increase in hardware complexity. The basic 
design of all the fast correlators is a multiply circuit and an adder 
tree. In the one-bit case it is possible to "hang" additional multiply 
circuits on the adder tree without effecting the propagation times 
considerably. The full adder is in fact a ternary function. Figure 36 
shows the logic diagrams for a 15 x 1 correlator. Figure 3/ is the 
extension of Figure 30 that produces a 15 x 16 correlator. This basic 
block is iterated in space to form the various array sizes listed in 


Table 9, 
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Figure 36. Logic diagram for a 15 x 1 one-bit correlator. 
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Figure 37. Extension of Figure 36 to produce a 
15 x 16 correlator. 








i 
: 
1 
; 
i 
i 
' 
{ 
i 





RA 
4 
| 

4 
eu 

$ 

a 
5 
eo 
of 
Py 
ey 
F 
BH 
BI 

ry 

& 
i 


7 
i 
q 
FA 
4 
FN 
& 
4 








Array Size 


Table 5. Qne-bit by one-bit correlator using 
ECL logic. 





It appears unlikely that a complete ECL implementation would be 
practical due to the large power requirement. In this section we will 
first describe an implementation based on low power Schottky TTL-logic. 
Since this logic family 1s not fast enough to do an addition of 2048 
(32 x 64) one-bit numbers in anything appruaching 133 ns, it is 
necessary to perform the computation in pieces. ‘his technique is called 
pipelining. Ouring a giver time interval the 'pipe' may be computing 
Part 1 of Item 3, Part 2 of Item 2 and Part 3 of Item 1; at the next 
time step computation is performed on Part 1 of Item 4, Part 2 of Item 
3, Part 3 of Item 2. A new result is generated at eachtime interval but 
for any specific data item the time required is the product of the basic 
time interval and the number of sections of pipe (see Figure 38). The 
Schottky nipe version illustrated on the accompanying pages (Figure 39) 


requires approximately 1600 packages and 300 Watts for a 32 x 64 array. 


: 
; 





P, Po P, 

L, time 
a; + | 
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I, 1, I, I, done 

I, I, I, I, done 


Figure 38. Items in pipe. 


The pipe delay in each of the three sections of pipe is 132 ns. 
Memory required in addition to that contained in the correlator itself 
fs assumed to be a charged coupled device since this will provide the 
necessary speed with very low power consumption. Bubble memories may 
become practical in this application as well. Assuming a picture image 
P x Q with R points devoted to horizontal retrace and sync, and a corre- 
lation array K x L, from Figure 40 we observe that the delay required is 
(Q+R-L) for each row. Using CCD (charge coupled array) power dissipa- 
tion at 10 MHz is about 20 uW/BIT. Therefore total power dissipation is 
20 x 10°3k(Q + R - L) milliwatts. 
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(Q+R- L) Bits each row 
K(Q + R - L) Bits total 


{ 
Figure 40. Memory requirement for correlation computation. 


Another problem involving the one-bit by one-bit correlator concerns 
the removal of the DC component of the LR image. This forms a support 


component in the system block design. The following is a discussion of 


a sub-optimal technique that accomplishes the removal of the OC digitally. 


Referring to Figure 41, in approximating the level about which to quantize 


pixel A the value of pixel B must be known. Pixel B lags A by (K/Z)(Q+R) + 


L/2 sample intervals. The column containing pixel B is used to determine 





Figure 41. Method of computing mean video level. 
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L mean values and then is not used until] Q + R - L events later. Oata 


Item C must be removed from the column sum after K(Q +R) events. The 


If we assume a 3 bit 
quantization of the picture, then 3 x [K x (Q +R) + 1/2] + 14x (Q+R +L) 


bits of memory are required As before, using Q + R = 476, K = 32, L = 64, 
53,352 bits are required, 


resulting algoritha is presented as Figure 42. 


Power dissipation is approximately 1.5 watts 


for the memory array using CCD. Clocking and interface logic could increase 
this number significantly. 


Mean Value 
(Shift to divide 






Comparator 


3x (K(Q + R) + L/2) 


+ lx (9 +R +L) Bits of memory 


Figure 42. Mean video computation and quantizer. 
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serves to reduce the hardware memory “equirements. In order to achieve : 


reasonable computational speeds through the adder tree, the circuit is 


far he 


i 
| 
{ 
i 
The use of half-frame correlation (i.¢., non-interiaced images) ¢ 
| 
| 
divided into sections (f.e., pipelined). With interlacing, one of the | 
final stages of the pipe sums results delayed from the previous half- 
frame. Because this delay requires considerable memory and because 
REECE EES RET TOE LESAN SOL SEO OLE ES PCI TEI TEI EI QDI DOTS  EI PE IE SDT AL TO SEP AER EE TREE 
of the effects of vibration discussed earlier, it is recommended that 
LL LSS OS ASS SSD SES SER 
a ha, f-frasie correlator be implemented, thus, effecting a cost, space 
and power saving. | 


The maximizing circuit is quite simpie and is shown in Figure 43 . | 
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5 The position counter is also used in the edge correction problem. The 


picture is dimensioned as in Figure 44 . ‘lotice that both the hori- 


bt ah A atv eter a en ws 


zontal and vertical blanking intervals are indicated by a one in the 
most significant bit of each counter. The retrace intervals have been 
adjusted to ensure this simple hardware implementation. The final 


construction may require that sample rates be adjusted to achieve this 
effect. 
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The mean value of the image needed for quantization near the edges 
is difficult to calculate because part of the K X L subarray extends 
into the horizontal retrace region or vertical retrace region as shown 
in Figure 45. The dc component of these areas that depend on values 


outside the field of view can be calculated by substituting a binary 
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noise source for the retrace voltage. One possible implementation is 


shown in Figure 46. This will preclude the possibility of false con- 


appears gray. 


1 
trast changes near the edge by smearing the edge so that it always 
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P x Q is single scan picture area 
For example 256 x 512 


R and S are horizontal and vertical 
blanking intervals 


For example (P + S) x (Q + R) could 
be 320 x 576 


Figure 44. Picture size (P x Q) with horizontal and 


vertical retrace R and S. 
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— no picture in these regions 





Figure 45. Retrace ‘Edge) effects on quantization. 
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Figure 46. Binary noise source generator, 
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The high resolution image contains far too much cata to be used in 
the correlation. This data is reduced by computing a spatial average 
using Equation 67 where Wi, and Wy are assumed to be integers. The com- 
ponents of the spatial avaraging are then sampled to reduce the data. 
Figure 47 illustrates the hardware for such 2 reduction. The digital 
approach uses MOS or CCD shift registers for delay. The precision of the 
HR image is assumed to be 3-bits. A binary adder with a register acting 
as accumulator forms the desired sums. The sum of 25 3-bit sumbers will 
not exceed 256 or 8 bits in precision. The Divider is a 2048 x 1 ROM. 
3x4, 3x5, 4x3, 4x4, 4x5, 5x3, 5x4, 5x5 horizontal and vertical reductions 
in the aspect ratios can be handled. The ROM, also causes the 3-bit 
precision of the image to be quantized to 1-bit. 

The horizontal and vertical blanking signals are used to force the 
magnitude comparator in the peak detecting circuit to ignore the corre- 
lation values computed during horizontal and vertical retrace. Since 
the correlator is designed to run free, these correlation values must be 
ignored during retrace. The circuit used to accomplish this is shown in 
Figure 48. Also illustrated is a method for selecting either minimum 
or maximum. If B is to be larger than A to track maximum, then the 


blanking should force B > A. 


B. Block Diagram of 32 x 64 Array Correlator System 
A detailed block diagram of a total correlator system has been drawn 
by incorporating all of the block diagrams of the previous figures. The cor- 


relation system is designed around the XOR implementation. The block diagram 
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Figure 48. Blanking circuits for correlator during 
horizontal and vertical retrace. 


also shows the pre-processing that is done on botir the HR and LR video 
signals assuming an adjustable FOV ratic between the two and also shows 


how the processed signals are clocked into the correlator. 
The completed implementation is shown in Figure 33. A more detailed 


representation is shown in fold out Figure 49. It is important to remember 
that the diagrams represent potential ways of implementing the desired 
design and are presented as a feasibility demonstration. A practical 
implementation might not employ identical techniques. The following 
discussion treats the blocks in Figure 33 as a system. The component 


functions were treated separately earlier in this chapter. The high 
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resolution signal is separated into video and sync components. The 
vertical sync is used 7s a reference to a phase-locked loop circuit (PLL) 
to provide the HR clock which is used to control the HR sample interval. 
The HR video is conditioned by a high pass fiiter to remove the mean or 
dc component of that signal. The HR signal is then spatially averaged 

in a field of varying size to reduce resolution to that compatible 

with the LR image. Selected pulses from the HR clock (again to reduce 
resolution) are used to load the HR portion of the correlator. The HR 
buffer is not loaded on every HR frame but at a slower rate. This rate 
may be target dependent. The LR sync is used as a reference PLL circuit 
in a manner similar to the HR sync. DC removai is accomplished on a 
pixel basis by using an approximate local averaging technique. This 
processed video is sent to the LR correlator buffer. The correlator 
produces correlation signals at the LR clock rate. The peak of this 
signal is passed to a peak averaging circuit that reduces the probability 
of false registration. The y coordinate is adjusted on odd frames. If 
the maximum detector fails to find a maximum above the specified threshold 


in a specified number of consecutive fields, the search mode signal is 
activated in an attempt to locate the target. The correlator speed is 


the limiting factor in this implementation. 

The speed of the correlator is limited by the fan-in at each step 
of the adder tree. The investigators have recently learned of an 
integrated circuit available from TRW that forms the XOR of 64 pairs of 
numbers and sums the resulting products by analog means. (The summing 


is done by adding currents at a node.) The speed of this device is quoted 
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E in the 5-10 MHz range. It may be possible to employ this device in a 


correlator implementation that does not require pipeline techniques. The 
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power requirements for this device are also well below the full digital 
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but the system should perform well in either approach. The investigators 


recommend the analog approach in view of present available information. 
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implementation. The analog adder is less precise than the digital one 
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5. CONCLUSIONS AND RECOMMENDATIONS 


Most of the conclusions and recommendations resulting from the 
work on this contract have been given and justified within the first 
four chapters of this report. For compactness they are given again in 


this chapter. 


A. Conclusions 


1. A one-bit by one-bit correlator using exclusive OR gate or exclusive 





NOR gate realization should be used in a hand-off technology program. 


2. The one-bit correlator may be either an implementation of the 


Pena eee 


Direct Summation Method (search for a maximum) or the Sequential 

E Simularity Detection Algorithm (search for a minimum) since they | 
a are equivalent in a one-bit by one-bit correlator. | 
’ | 3. Ina technology program the reference array size should be variable | 
in order to investigate the trade-off of reference array size vs. 
: correlation accuracy using typical military targets and scenes. 

: The size of the reference array in the correlator should be variable 

: up to at least 32 x 64 or 2048 elements. The statistical reliability 

: | of the correlator improves as the number of elements increases. 

4 | 4. The adder tree may be either digital or analog. The digital adder 

E is more accurate but requires more hardware than the analog imple- 

F mentation. There exists more clock timing problems with the 

F 


digital adder tree. 
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The exclusive OR gates in the correlator should be in special 
purpose chips, like the one developed by TRW and reported in 
Chapter Four of this report. These special chips reduce the 
amount of hardware significantly. Other companies may manufacture 
similar chips but they are unknown to these investigators. 
The 32 x 64 correlator should operate on 32 succeeding lines in 
each TV field rather than 16 lines from one field and the 16 
interlaced lines from the next TV field. This implementation is 
referred tu as a half-frame correlator and reduces the required 
hardware and minimizes the effects of vibration. 
Some form of image normalization is required to prevent false 
registration points. For the one-bit by one-bit correlator no 
normalization is required since the images are already normalized. 
If a higher level quantizer is desired, there is not much point 
in using more than four levels from a S/N point of view. If more 
thar, two levels are used, care should be taken in how the quantiza- 
tion step levels are obtained. Inaccuracies in the quantization 
levels between the two video signals could cause a larger reduction 
in the S/N ratio than that gained by going to more levels. Using 
more than two levels can cause further degradation because fewer 
picture elements can be processed in a given time. 
At the ranges at which the correlator will be operating, it is 


actually correlating on an area and not the target since the 


target represents only a small portion of the reference image. 
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10. With a 32 x 64 or 2048 pixel reference array, the performance of renee 


CRT ETT 
aT TEESE Cog 


the correlator will depend almost entirely on the goodness of the 


y preprocessing of the two images. The various preprocessing design 
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parameters should be investigated in a technology program. Realistic a E 


sensor differences should be included in this investigation. 
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11. The search pattern shown in Figure 32(b) is recommended if the 
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target is not found in the initial missile seeker FOV. _ 
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4 B. Objectives of a Technology Program 
4 From the results of this study it is concluded that hand-off of a 
F target from a pointing and tracking system to a missile seeker using a 
4 correlator is not only feasible but an attractive option. It is 
; recommended that a correlator be built and used in a technology program 
| ’ to demonstrate this feasibility and to study the various trade-offs 
F discussed in this report. The following are some recommendec objectives 
: of such a technology program. : 
E 1. Determine overall performance of correlator. Performance 

: criteria include average time for target lock-on, percentage | 

: of false lock-ons, percentage of missed targets, etc. . 

E 2. Investigate the accuracy of the correlator vs. the size of 

E reference array used. This might be scene dependent. 

: 3. Getermine the optimal threshold value to use in the correlator. 

, This is also scene dependent to some extent. 

F 4. Determine the effects of varying the TV signal sampling rate 
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from approximately 3.5 MHz to 8 MHz. 
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Investigate sensitivity of correlation algorithm to quantization 
about incorrect dc level. 

Investigate sensitivity of correlation algorithm to errors in 
Wi, ax Wy in reducing the resolution of the PTS video to 

that of the seeker video. 

Investigate effects of vibration on the correlator accuracy. 
Determine sensitivity of correlator to sensors with differing 
spectral responses. 

Determine rate at which HR reference image must be up-dated 
with typical moving targets. 

Investigate sensitivity of correlator to various video S/N 


ratios. 
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